mindforger
mindforger copied to clipboard
dvorka
Headers aren't rendering
whenever I add a header tag (#), it doesn't get rendered in the preview. Also, whenever I return to edit the file I find that the '#' is moved over one space. Being that the css files support headers (so therefore the renderer is probably not the issue), is mindforger "subconsciously" moving every header tag of a space?
TL:DR use menu Note/New to create a new section - "manual" section within section is intentionally quoted.
MindForger uses Markdown as a format for storing data, but it aims to do more. Therefore it splits Markdown file to sections and represents each section as Note:
- MindForger shows Notes outline - tree of Notes on the left in Notebook view
- You can manipulate with sections (up/down/promote/demote/clone/refactor/...) from menu
Note - It offers associated sections (Notes) to the section being read/written
- ...
See also 
That's all good and fine, but why make the sub notes separate from the main doc and why not auto generate sub notes, when you add a header to a document.
P.s. thanks for the fast response.
Quoting feature is based on user feedback and obviously it's an extra code (doing no quoting would be obviously simpler). Typical use case is copy-paste of a shell or Python source code w/ comments - if user forgets to quote it, then it breaks Markdown document structure.
However, I understand why you would like to avoid quoting - let me make this configurable feature #668 i.e. it will be possible to switch quoting off.
I think two things should happen:
- Whenever you type a single header
#, the system should auto add a sub-note, but not separate it from the main doc. If you type any other header that would be either a sub-note within a sub-note or nothing special at all. - If you add a sub-note, it would add a header to the current note. if you add a sub-note to a sub-note (up to 4 times) it would add a lower level header (i.e.
h2throughh5).
I just bumped into this. I recently started using Mindforger for documentation purposes at work. We use markdown in an internal tool, so my aim is to write locally in MF and then upload it to the tool when ready.
However, when trying to see how I could get linking to work to a specific h2 in a note, I noticed that all h2 ('## Title') I had written on each note were showing up indented (' ## Title'). I purposefully did not use h1 ('# Title') in my notes, as I leave that to the note title. As such, the live preview on another editor (ReText) was broken. Only the title of each note shows up correctly. All other headers are indented, hence shown as a normal text line.
While I would love a configuration option for avoiding any inserted indentation at all (I do not know how MF recognizes notes but I see metadata content on the title lines for each note, so I'm guessing it should be possible), I do not mind as long as I'm working in MF, as the headings are readily recognized in both the full notebook view as well as each individual note view. Another great feature (for which I can open an issue if needed) would be to have an export function that exports the entire notebook in its .md format, without quoting or added metadata, that could be readily imported or pasted into another system that recognizes markdown.
Note: I did not convert each h2 into its own 'demoted' note (section/subsection) because they are quite short ones, grouped around a single concept (reflected in each note's title) and it's easier for me to edit that note in full rather than editing each paragraph separately. This, of course, is subject to YMMV.
I'm also all for MF automatically recognizing headings as subnotes, as long as I can edit a note and have all its headings (subnotes) available to me.
Edit: I've also noticed that the indentation seems to be applied also to lines starting with '#' inside code blocks (either ~~~ or enclosed in 'pre' tags).
@godlike64 thank you for the comprehensive description of the issue. Can you please add a few example to your text? It would help me to make sure that I understood 🐞 , something like:
# Title 1
Description.
# Foo title 1.1
## Foo title 1.2
# Title 2
... would be perfect. Thank you!
Hey @dvorka
Thanks for your fast response! Sure, I can provide more info.
First of all I want to clarify that I know only the bare minimum of C++ (if MF was written in Python you'd already have a pull request or two ;).
As for examples, here is the code preview bit I first used:
<pre>
# man <i>function_name</i>
</pre>
This is used to indicate a command that should be run as the root user, hence the leading '#'. Problem is that MF applies the leading space to that one too:
<pre>
# man <i>function_name</i>
</pre>
Hardly noticeable inside a code block, and still valid markdown so other than the cosmetic issue of a line in a code block not starting on the edge, not really a problem.
The real problem is if I'm writing the following note. Let's say I'm talking about the files of a program or library, so I have a Note called 'Files'. In the notebook full view, this 'Files' gets rendered as an h1, which is good. Then inside this note, I am going to be listing some files with a short description for each, so I (again, me personally but YMMV) see no point in splitting this into a different subnote, as I much rather wish to look at what I'm writing about all files to see that I'm making sense. The section would look something like this:
These are the files relevant for application_x
## /path/to/file.conf
This is the configuration file. It has INI-like syntax and you can open it in any text editor.
## /path/to/file.log
This is the log file. It is a text file where all application_x's events are written to.
## /path/to/file.pid
This is the PID file. It contains the PID of application_x and is used when restarting to send the SIGTERM to the correct process.
Now suppose, for my work, that I'm gonna have a webpage in the markdown tool for the different sections of the document I need to upload, and one of those sections is gonna be 'Files'. Then I would just copy and paste from the edit view of MF. The problem is that after saving and reloading the MF note, it now looks like this:
These are the files relevant for application_x
## /path/to/file.conf
This is the configuration file. It has INI-like syntax and you can open it in any text editor.
## /path/to/file.log
This is the log file. It is a text file where all application_x's events are written to.
## /path/to/file.pid
This is the PID file. It contains the PID of application_x and is used when restarting to send the SIGTERM to the correct process.
Since there's a leading space before every h2, they will not get the proper markdown syntax applied to them and will simply show up as a normal plaintext line. Before uploading the document, I would have to manually remove the leading space on all h2 lines so it gets uploaded and shown correctly.
I did manage to find out where this leading space was being written:
lib/src/representations/markdown/markdown_outline_representation.cpp:
342 void MarkdownOutlineRepresentation::description(const std::string* md, std::vector<std::string*>& description)
343 {
344 if(md) {
345 istringstream is(*md);
346 bool codeblock=false;
347 static const char SECTION = '#';
348 static const char CB = '`';
349 for(string line; std::getline(is, line); ) {
350 // stupid and ugly hack: TAB all lines starting with # to ensure correct sections separation
351 // (user creates such line when edits N description)
352 if(line.size()>=3 && line[0]==CB && line[1]==CB && line[2]==CB) {
353 codeblock = !codeblock;
354 }
355 if(line.size() && line.at(0)==SECTION && !codeblock) {
356 line.insert(0, " ");
357 }
358
359 // TODO add quoting of multiline sections that use === and ---
360
361 description.push_back(new string{line});
362 }
363 } else {
364 description.clear();
365 }
366 }
I simply replaced line 356 to read:
line.insert(0, "");
I recompiled and opened up MF, removed all the leading spaces from my notes, saved and reloaded each note and the leading spaces were still gone. Now, I closed and reopened MF and then each h2 had been put into its own section. No further metadata was added to the h2 in the raw .md file, but MF recognized h2 as a demoted note (a subnote inside a note) so it arranged them like so. Problem was that the code block (the 'man function_name') was also recognized as a section, splitting part of a note in two. I rolled back the change and reopened MF but it did not fix this note reorganization, even though I found no metadata was added to each header. I had to rewrite my notes (i.e. take the content of the demoted notes and put it back into the main note) and delete the subnotes.
Now, again I don't know much C++ so I'm not familiar with how MF was built, but from what I see it looks like lines starting with '#' will always be considered a place where metadata should be read, and be used to determine where a note/subnote starts/ends. Perhaps it would be better not to rely on whether a line starts with '#' for this check, and instead see if the line contains a certain string (such as "<!-- Metadata: type:")? That way there would be no need to add a space if the line starts with '#', and we could let the user choose when to instantiate a new note, letting him/her use any markdown syntax on each note, while also ensuring interoperability with other markdown editors.
Do let me know if this makes sense or if I'm mistaken. And BTW I already can't see myself using any other thing than MF for markdown editing :D I loved the IDE approach it has.
Edit: hmmm, I did some further digging and apparently markdown parsing is not an exact science, to say the least :) Looks like ReText completely refuses to recognize a line starting with ' #' as a heading, no matter the level. However, KMarkdownWebView (a KPart for KDE that integrates markdown syntax highlighting and a live preview pane on any text editing application that uses the Kate framework) seems not to care at all, unless there are like 4 leading spaces, at which point it stops rendering it as a heading. I then went on to an online markdown editor which similarly didn't care at all and rendered the heading without issue with 1-3 leading spaces (I'm starting to think the break at 4 is because of the 4-space-indent rule in markdown).
But of course, the tool we use at work refuses to parse the heading with any leading space before it. That is unless the previous line was part of a list (started with '- ' or '* ') in which case, the heading gets parsed normally.
@godlike64 thank you for the comprehensive response! I really appreciate it. I think that I understand the problem now:
- MindForger uses Markdown format for storing documents, in particular GitHub Flavored Markdown
- There is specification which defines GitHub Flavored Markdown syntax
- MindForger adds HTML comments at the end of section titles to maintain document and section metadata
- It is valid construction and it ensures that metadata will be hidden as Markdown is converted to HTML
Regarding sections, # and ##:
- Markdown specification defines that every row which starts with one or more
#characters is section -
#is not identified as section marker if:- It is in code fence (in a block which starts and ends with 3 backticks OR alternatively it might be prefixed with 4 (or more) tabs which is another syntax
- This is why MindForger escapes lines which start with
#within the section by prepending space (they would become sections on such document reload).
What to do next?
- Line starting with
#will be always considered section because of Markdown specification. - You can use the following syntax to legally show preformatted text:
- either use ``` based code fence to create pre-formatted block
- or create code fence using indentation
- or create pre-formatted block using
<pre>(Markdown to HTML conversion will skip it) - or represent # using HTML entity
#
Any suggestion, idea or preference?
- I can improve / change MindForger to use escaping of
#rows. - Instead of pre-pending space MindForger could be configured to use different escaping (convert # to HTML entity, add line with backticks before and after #, ...) - what would you prefer?
Thank you for your interest in MindForger, sharing how you use it and what are the problems!
I can improve / change MindForger to use escaping of # rows.
This is something I'd also like to see.
Instead of pre-pending space MindForger could be configured to use different escaping (convert # to HTML entity, add line with backticks before and after #, ...) - what would you prefer?
Instead, how about minforger will enclose notebook and note headers with #, and leave other headers as is?
# Notebook 1 # <!-- Metadata -->
# Note 1 # <!-- Metadata -->
## Part of note 1
Note...
## Another one
Note...
# Note 2 # <!-- Metadata -->
etc.
Or perhaps using metadata for headers within notes too:
# Notebook 1 <!-- Metadata -->
# Note 1 <!-- Metadata -->
## Part of note 1 <!-- Metadata: Type: Note header; -->
Note...
## Another one <!-- Metadata: Type: Note header; -->
Note...
# Note 2 <!-- Metadata -->
etc.
MindForger could be configured to use different escaping (convert # to HTML entity, add line with backticks before and after #, ...)
If I were to just choose from given options, it would be "convert # to HTML entry".
Though, converting note and notebook headers would look better:
<h1 metadata="..">Notebook 1</h1>
<h1 metadata="..">Note 1</h1>
## Part of note 1
Note...
## Another one
Note...
<h1 metadata="..">Note 2</h1>
etc.
Thanks for the explanation @dvorka. I want to clear up that my problem is not with MF itself: I have been able to more or less make it suit my needs, and as long as I don't leave MF, I have no problems with the format.
The issue is when I have to export from MF to another tool that accepts Markdown. As it stands, while I have a way to view the entire notebook by clicking on the notebook title (which is perfect to perform a general review of where my document stands), there is no way to access this in a pure markdown format, other than accessing the actual .md file in the filesystem. The problem is, due to how MF escapes headers, and how the tool I must use interprets those ' #' lines, I would have to first process it (either manually or using a script) so that the entire notebook fits the syntax to be copy-pasted into the tool and be recognized.
While this is also not a big problem (for me, at least), since I can easily write up a Python script that removes leading spaces before headers, it would be nice if we could have an export option for pure markdown. It doesn't need to be for specific notes (although that would also be nice), it can be for the entire notebook. However, it would need to at least remove the leading spaces to ensure the best compatibility with all markdown editors (note, as before, that ReText also refuses to parse ' #' lines).
As a side note, lines inside a code block that start with '#' are also being escaped by prepending a space.
Overall, i'd think the first or second solutions proposed by @akinokonomi would be best: let MF parse only <!-- Metadata blocks in order to determine notebook subdivisions and anything else necessary for MF to work. It would of course need a bit of refactoring, since you are no longer looking for lines that start with '#' but rather lines that contain '<!-- Metadata'. The best thing about this is that I am free to write markdown by hand, while also having the freedom to choose when to create a subdivision (note/subnote), and keeping the raw file suitable for copy-pasting into any other Markdown editor.
And if you put either of those two solutions together with a Markdown export function? Give me your 'Donate' button! :-D
Remarks:
- Any line starting with
#and not in code fence is section by definition ~ Markdown specification (atx-style). Therefore I unfortunately cannot show sections based on metadata hint as it would violate the spec and defined rendering rules. It would also confuse users who expect such behaviour.
Examples of escaping (1.54.0 development version):
-
Text written by user before saving Note i.e. before escaping:
-
Text escaped using HTML entity - rendered as normal text:
-
Test escaped using four spaces - rendered as code fence:
Testing:
- escaping of lines starting with
#within code fence is not reproducible - line is left intact
Next steps:
- [x] let user choose between code fence and HTML entity in configuration
- [x] add ^ to File/Preferences dialog/Markdown tab dialog
- [x] remove ' ' escaping as it doesn't make much sense
@godlike64 which export formats would you like me to implement? Do you want to export all files, Notebook or Note?
Hey @dvorka
Just to confirm that we are talking about the same thing, because maybe we are not and hence the confusion. Plus, probably neither of us speaks English as a native language, which makes things harder :)
What the markdown specification defines in lines starting with one or more '#' is a header (as in, the h1, h2, ... hN HTML element it gets converted to).
What I refer to as section are the lines that mark the beginning of a note (and thus the end of the previous one). Lines like:
which just so happen to be an unescaped h1 header with the metadata block. MF uses this to determine when a note ('section') ends and the next one begins, which just so happens to be rendered as an h1 in the final document. But, from what I understand by looking at the markdown specifications you shared, this is arbitrary: there is no definition in the Markdown specification for this concept of a section. This is strictly something that MF implements and what makes it unique (and please do correct me if I'm wrong and I'm missing something here).
What I'm saying is that:
- The need for a section ('note' in MF terms) beginning with an h1 is arbitrary. A section could begin without this h1, and MF could realize where a section begins just by looking at the metadata block. For example, if I wanted all my sections (notes) to begin with an h2, because I want to have only one h1 in the final document, this is not possible, as MF will always render the note title as an h1. This is what I meant in my previous comments.
- Right now, MF prepends a leading space on headers of any type (from h1 to h6) so that they do not get parsed as beginnings of sections (notes).
- Prepending a leading space on lines starting with '#' happens even if we are inside a code block (this definitely should not happen).
Perhaps this proposal should be a different issue, marked as enhancement on its own. I can try to cook up something on my own (no ETA since I don't know much C++) and if I'm successful, i can definitely submit a PR for reviewing!
As for the export format, it all boils down to avoiding what I wrote above, so that I can directly copy and paste from an exported markdown file onto another editor (that might not recognize lines starting with ' #' as headers). It doesn't need to be on a per-note basis, it could just be the entire notebook, without the leading spaces that MF inserts.