HTTP 500 when creating an object with our markdown test content from project pages

[amoeba]

While testing out Markdown support, I ran into this:

> createObject(mn, pid = sm@identifier, file = eml_path, sysmeta = sm)
Error in .local(x, ...) : 
  Error creating 7af9e782-8220-4f23-90e7-0b56b36ba077: Server error: (500) Internal Server Error

The document I uploaded is at the bottom of this issue. I don't see anything super helpful in catalina.out but I did find that removing about the last 300 lines from the markdown allowed the document to be created.

I do see lines from catalina.out about DBSAXNode.writeChildNodeToDBDataLimited - SQL insert: INSERT INTO xml_nodes which makes me think building the xml_nodes tree for the doc might be involved.

I want to find a stack trace first so I'm going to do that and then follow up here.

eml.xml

Mr. Clarence Lehman Citations

This is an example of some inline citations [@jones_2001]. The citation information should be extracted from the literature cited section [@brinckman_2018, @collins_2018].

The purpose of this example is to test the features of showdownjs.

Features tested: {toc}

• [x] Inline HTML
• [ ] Docbook Tags
• [x] Automatic Paragraphs
• [x] Headers
• [x] Blockquotes
• [x] Lists
• [x] Code Blocks
• [x] Horizontal Rules
• [x] Links
• [x] Emphasis
• [x] Inline Code
• [x] Images
• [x] Fenced Codeblocks
• [x] Syntax Highlighting
• [x] Tables
• [x] Metadata
• [x] URL Autolinking
• [x] Custom IDs on Headlines
• [x] Footnotes and Other Links more testing requried
• [x] Todos
• [x] Strikethrough
• [ ] Definition Lists not supported
• [x] Math more testing requried
• [x] Citation-references more testing requried
• [x] Bibliography formatting more testing requried
• [ ] XSS filter DIY?
• [x] Bootstrap classes
• [x] Emojis
• [ ] Citations
• [ ] Math

Extensions required:

Working:

• DIY: showdown-highlightjs
• DIY: showdown-bootstrap

To be tested in metacatUI:

• citation-js-showdown
• showdown-katex
• showdown-footnotes

To create?

• showdown-xss-filter

Inline HTML

This is surrounded by `h3` tags

This is surrounded by `strong` tags

back to table of contents

Docbook Tags

This is the example text...

<para>This line and the next one are surrounded by `para` tags</para><para>This should also be a paragraph. Here is an example of a <subscript>subscript</subscript></para>

This is how it is rendered:

This line and the next one are surrounded by para tagsThis should also be a paragraph. Here is an example of a subscript

The original tags are preserved in the HTML, but are not styled or converted to html.

---

Some more docbook tags:

citetitle emphasis ulink with url

Literal Layout Hope this will Preserve White space

    </literallayout>

itemizedlist item one itemizedlist item two ordered list item one ordered list item two subscript superscript

back to table of contents

Automatic Paragraphs

A paragraph is created by using

two

linebreaks

back to table of contents

Headers

If the above "Headers" text is large, then headers work.

back to table of contents

Blockquotes

This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote. This is a block quote.

back to table of contents

Lists

Showdown supports ordered (numbered) and unordered (bulleted) lists.

Unordered lists

You can make an unordered list by preceding list items with either a *, a - or a +. Markers are interchangeable too.

* Item
+ Item
- Item

Ordered lists

You can make an ordered list by preceding list items with a number.

1. Item 1
2. Item 2
3. Item 3

It’s important to note that the actual numbers you use to mark the list have no effect on the HTML output Showdown produces. So you can use the same number in all items if you wish to.

List syntax

List markers typically start at the left margin, but may be indented by up to three spaces.

   * this is valid
   * this is too

List markers must be followed by one or more spaces or a tab.

To make lists look nice, you can wrap items with hanging indents:

*   Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
    Aliquam hendrerit mi posuere lectus. Vestibulum enim wisi,
    viverra nec, fringilla in, laoreet vitae, risus.
*   Donec sit amet nisl. Aliquam semper ipsum sit amet velit.
    Suspendisse id sem consectetuer libero luctus adipiscing.

But if you want to be lazy, you don’t have to

If one list item is separated by a blank line, Showdown will wrap all the list items in <p> tags in the HTML output. So this input:

* Bird

* Magic
* Johnson

Results in:

<ul>
<li><p>Bird</p></li>
<li><p>Magic</p></li>
<li><p>Johnson</p></li>
</ul>

This differs from other markdown implementations such as GFM (github) or commonmark.

Nested blocks

List items may consist of multiple paragraphs. Each subsequent paragraph in a list item must be indented by either 4 spaces or one tab:

1.  This is a list item with two paragraphs. Lorem ipsum dolor
    sit amet, consectetuer adipiscing elit. Aliquam hendrerit
    mi posuere lectus.

    Vestibulum enim wisi, viverra nec, fringilla in, laoreet
    vitae, risus. Donec sit amet nisl. Aliquam semper ipsum
    sit amet velit.

2.  Suspendisse id sem consectetuer libero luctus adipiscing.

This is valid for other block elements such as blockquotes:

*   A list item with a blockquote:

    > This is a blockquote
    > inside a list item.

or even other lists.

Nested lists

You can create nested lists by indenting list items by four spaces.

1.  Item 1
    1. A corollary to the above item.
    2. Yet another point to consider.
2.  Item 2
    * A corollary that does not need to be ordered.
    * This is indented four spaces
    * You might want to consider making a new list.
3.  Item 3

This behavior is consistent with the original spec but differs from other implementations suck as GFM or commonmark. Prior to version 1.5, you just needed to indent two spaces for it to be considered a sublist. You can disable the four spaces requirement with option disableForced4SpacesIndentedSublists

To nest a third (or more) sublist level, you need to indent 4 extra spaces (or 1 extra tab) for each level.

1.  level 1
    1.  Level 2
        *   Level 3
    2.  level 2
        1.  Level 3
1.  Level 1

Nested code blocks

You can nest fenced codeblocks the same way you nest other block elements, by indenting by fours spaces or a tab:

1.  Some code:

    ```js
    var foo = 'bar';
    console.log(foo);
    ```

To put a indented style code block within a list item, the code block needs to be indented twice — 8 spaces or two tabs:

1.  Some code:

    var foo = 'bar';
    console.log(foo);

back to table of contents

Code Blocks

This is a block of code

back to table of contents

Horizontal Rules

Here are how three dashes (---) with a line break before and after are rendered:

---

back to table of contents

Links

Simple

If you wrap a valid URL or email in <> it will be turned into a link whose text is the link itself.

link to <http://www.google.com/>

this is my email <[email protected]>

In the case of email addresses, Showdown will also perform a bit of randomized decimal and hex entity-encoding to help obscure your address from address-harvesting spambots. You can disable this obfuscation setting encodeEmails option to false.

With the option simplifiedAutoLink enabled, Showdown will automagically turn every valid URL it finds in the text body to links for you, without the need to wrap them in <>.

link to http://www.google.com/

this is my email [email protected]

Inline

You can create an inline link by wrapping link text in brackets ( [ ] ), and then wrapping the link in parentheses ( ( ) ).

For example, to create a hyperlink to github.com/showdownjs/showdown, with a link text that says, Get Showdown!, you'd write this in Markdown: [Get Showdown!](https://github.com/showdownjs/showdown).

Reference links

this is a [link to google][1]

[1]: www.google.com

this is a link to [google][]

[google]: www.google.com

back to table of contents

Emphasis

bold and italics

back to table of contents

Inline Code

Use single backticks () to format text in a special monospace format like this`. Everything within the backticks appear as-is, with no other special formatting.

back to table of contents

Images

Inline image syntax looks like this: ![Alt text](url/to/image). For example:

back to table of contents

Fenced Codeblocks

# This has already been demonstrated, but here is another example.

library(SomeLibraRy)

this %>%
    is() %>%
    fenced() %>%
    code()

back to table of contents

Syntax Highlighting

In this example, show down has been extended to use highlightjs. Highlight js has plenty of style sheets to choose from, and can be customized to highlight most of the common languages (including R and Python).

Here's an example of some javascript:

var x = 5;
var str = 'str';
console.log(str);
/* a comment */

back to table of contents

Tables

This requires the option tables to be set to true here is a table:

Tables	Are	Cool
col 3 is	right-aligned	$1600
col 2 is	centered	$12
zebra stripes	~~are neat~~	$1

back to table of contents

Metadata

metadata: Metadata support is enabled when creating a new converter (new showdown.Converter()), by adding the argument metadata: true. Then information between --- and --- at the top of this document can be accessed using converter.getMetadata(). If successful, the information extracted will be displayed at the very bottom of this jsfiddle document.

back to table of contents

URL Autolinking

with the argument simplifiedAutoLink set to true in the new showdown.Converter(), text like www.google.com will be converted to <a href="www.google.com">www.google.com</a>. Like so:

You can search for things on www.google.com

back to table of contents

Custom IDs on Headlines {this-is-my-custom-id}

Check the console. This header should have the id this-is-my-custom-id. It works, but the dashes are removed. The option customizedHeaderId must be set to true in new showdown.Converter().

The custom id shows up, but with dashes (-) removed.

back to table of contents

Footnotes and Other Links

Some word or something that needs explaining[^1].

[^1]: The explanation.

[^1]: A single line footnote

[^5]: This is a paragraph.

That is another paragraph which is still within the same footnote.

back to table of contents

Todos

TaskLists (GFM Style)

Showdown also supports GFM styled takslists if the tasklists option is enabled.

 - [x] checked list item
 - [ ] unchecked list item

• [x] checked list item
• [ ] unchecked list item

Strikethrough

syntax:

~~strikethrough~~

~~This is a strikethrough~~

back to table of contents

Definition Lists

Not supported. But if a user really wants one, they can create it using the regular <dl>, <dt>, <dd> HTML tags. Like so:

<dl>
  <dt>SGML</dt>
    <dd>The Standard Generalized Markup Language</dd>
  <dt>HTML</dt>
     <dd>The Hypertext Markup Language</dd>
     <dd>The markup language you use to create web pages.</dd>
  <dt>XML</dt>
    <dd>The Extensible Markup Language</dd>
</dl>

SGML

The Standard Generalized Markup Language

HTML

The Hypertext Markup Language

The markup language you use to create web pages.

XML

The Extensible Markup Language

back to table of contents

Math

Math should be possible using the showdown-katex extension. However, it's too complicated to demonstrate using this JSfiddle environment (no CDN available).

back to table of contents

Citation-references & Bibliography formatting

The same goes for citations and biliographies, but we should be able to implement them using the citation.js-showdown extension.

back to table of contents

XSS filter

This is an important thing to consider because users will be able to input scripts like so:

<script>alert('xss filtering is required!')</script>

Showdown doesn’t include an XSS filter, so you must provide your own. See details of vulnerabilities outlined (here)[https://github.com/showdownjs/showdown/wiki/Markdown's-XSS-Vulnerability-(and-how-to-mitigate-it)]. There is a Showdown XSS Filter extension available, but when used in this fiddle, the docbook tags were converted to text, the tasklist (todo) checkboxes broke, the syntax highlighting stopped working, and the head ids were removed. You can see this in action by adding xssfilter to the list of extensions.

back to table of contents

Bootstrap classes

These can be added with a DIY extension, as outlined here.

The following classes were added automatically in this demonstration:

const classMap = {
  h1: 'ui large header',
  h2: 'ui medium header',
  ul: 'ui list',
  li: 'ui item'
}

back to table of contents

Emojis

This is not important, but emoji support is built in!

:+1: :tada: :smiley: :cat: :fried_shrimp: :computer:

more here extension to also allow font awesome icons here

back to table of contents

Math

block

&& x = (-b+-sqrt(b^2-4ac))/(2a) &&

E=mc^2

inline

Here is some $E=mc^2$ math $R^2=x(2*4)$ that is inline

  ]]></markdown>
</abstract>
<contact>
  <references>clarence.lehman</references>
</contact>

bar bar