Issue with markdown export if jrnl entry already contains markdown formatting.

[griffrawk]

Bug report

Hello! Thank you for reporting an issue! If you would fill out the below points, that would make our process a whole lot easier!

• Please tell us about your environment:

  • Jrnl version: (run jrnl -v) : v2.0.1
  • How you installed Jrnl : Arch AUR package https://aur.archlinux.org/packages/jrnl/
  • Operating system [MacOS, Linux, Windows?] : Archlinux

• What is the current behavior? If a line in a jrnl entry starts with # (hash, in this case from pasted in shell commands), when the entry is exported as markdown (jrnl -N --export markdown), the export adds H3 markdown tags to the beginning of the line.

• Please provide the steps to reproduce and if possible a minimal demo of the problem As the input and output contain markdown, they do odd things to this issue editor. So I have prepared and attached example files to this issue. I wrote the entries as a codeblock (3 backticks) so that a markdown reader should display the lines verbatim. However the markdown export changes the lines.

jrnl_entry_raw.txt jrnl_entry_markdown.txt

• What is the expected behavior? The lines beginning with # should remain unchanged.
• Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

[wren]

Confirmed on current develop branch (~v2.1.2). It looks like any line that starts with a # gets additional markdown added to it (even if it's in the body of the journal entry, which shouldn't have anything added).

[wren]

@griffrawk Would you mind walking me through installing from the Arch source above? I'm not very familiar with it, and none of the project maintainers know anything about it. In fact, it lists a different maintainer on there for some reason and the package is out of date. It looks like someone has been uploading jrnl to this repo, but I'm not sure who it is, or how to go about making sure it gets updated properly.

If I can get some more details about it, we may or may not be able to add it is an official distribution method (if enough people use it).

[griffrawk]

@wren Sure can.

AUR is the "Arch User Repository", a collection of unofficial packages for Archlinux that are built by Arch community users, not the offical repo maintainers. Any user can build a PKGBUILD which enables them to install any sort of package on their arch box under the control of the official package management system, pacman. The AUR exists as a public repository of those PKGBUILDs for others to use at their own risk. When one downloads PKGBUILDs for unofficial AUR packages it pays to do some due diligence on the contents! AUR packages can be voted on to choose what may be promoted into the official repos, as long as a 'Trusted User' can be convinced also to take it on.

Installing global site-packages in Arch can be a bit risky using pip etc. due to the risk of the official package tools not knowing of the existence of the files so the general rule is to use the arch native installers, hence why PKGBUILDs are made, or use virtualenv, or pip install into local site-packages instead for a single user, if appropriate. See this recent Reddit thread I made a few comments in for some more background https://www.reddit.com/r/archlinux/comments/ed7oka/aur_vs_custom_package_managers/

In the case of jrnl, a user has created an AUR page at https://aur.archlinux.org/packages/jrnl/, and made a PKGBUILD which references jrnl-2.0.1 from PyPI. One clones the relevant git URL and runs a build process (makepkg) to fetch the source and compile and install as appropriate. makepkg installs the source, using the rules in the PKGBUILD, into a 'fakeroot' environment and catalogs the files before producing a pacman compatible bundle, which is then installed using pacman. See https://wiki.archlinux.org/index.php/Creating_packages

The fact that the build references an old jrnl version is entirely the responsibility of the build maintainer who probably hasn't got round to a new version yet! If you were interested in taking on the long term promotion of jrnl in Arch, the best thing to do would be to get in contact with the AUR package creator and see if they'd be interested in passing the baton to you. You could then ensure the latest version was referenced in the build. Alternatively you could mention using a user-based install in addition on your quickstart page, in the case that users may be unable to install using pip to the global site-packages e.g. pip install --user jrnl.

For my own installation I just used the AUR package, but I could just as easily installed a local site-package using pip. However, based on your early comments I'd still see the bug even in the latest version!

I hope this is of some help.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[micahellison]

Back to the original issue:

I'm of two minds about this. The current markdown exporter assumes that anything in the entry that looks like markdown should be treated as such, and reformatted within the context of the export. It makes sense in the context of the larger export to turn top-level headings into fourth-level headings since the other tiers of headings are used to describe where we're at in the jrnl:

C:\dev\jrnl>jrnl
[Compose Entry; on a blank line, press Ctrl+Z and then Enter to finish writing]

Testing
# Here is my top-level heading.
Here is body text.
^Z
[Entry added to default journal]

C:\dev\jrnl>jrnl -n 1 --export markdown
# 2020

## March

### 2020-03-07 12:56 Testing

#### Here is my top-level heading.
Here is body text.

If we keep this convention, then the fix to the issue is to continue building out what is basically our own little markdown parser, and to be sure that markdown isn't reformatted when it's escaped with backticks like in the file @griffrawk attached to the issue.

But if we are exporting non-markdown text, such as PHP code snippets pasted into a journal entry, then we'd expect it to escape it. Instead, the current behavior, even with the above fix, would mangle it, turning # comment into #### comment instead of escaping it like \# comment.

In other words, should this be a markdown-to-markdown exporter, or a plaintext-to-markdown exporter? And if we keep it as a markdown-to-markdown exporter, we should mention that in the documentation.

[wren]

I don't think we should do any parsing of the body of any journal entry. For the markdown exporter, we should just add our headings, and then the body as-is.

I think building our own markdown parser is a rabbit-hole I would like for us to avoid.

[MinchinWeb]

I was the one who wrote this code several years ago.

The idea was to allow my journal to contain markdown entries that would flow nicely into a single file export, bumping up the headings to that export file could be displayed or rendered into HTML directly (which is exactly what I was doing at the time); much like michaellison's example. I work on a Windows machine, so my entries don't tend to contain Linux shell commands...

The matching code is a simple regex, to avoid trying to build a markdown parser.

As for possible solutions, it seems to me that there should be a an existing Markdown parser that can bump heading levels; I'm sure we're not the first to come across this issue. A quick search come across https://github.com/SaschaCowley/markdown-headdown which is designed to decrease heading levels, but should be be able to increase levels as well (configure with a negative offset?). However, this operates are as Markdown extension, so I don't know if this can be used directly to produce Markdown output, or if it will only output HTML.

Another solution would be to add a --raw tag to the Markdown exporter to sidestep this processing.

[MinchinWeb]

Also, if the heading code is updated or the raw flag added, it would like make sense to make the changes to both the Markdown and the YAML exporters.

[MinchinWeb]

Here is the discussion of the original implementation -->https://github.com/jrnl-org/jrnl/issues/335 ("Smarter handling of markdown headings in entry", May 19, 2015)

and the corresponding Pull Request --> #347