nested shortcode with codeblock rendering issue

[davidejones]

What version of Hugo are you using (hugo version)?

$ hugo version
Hugo Static Site Generator v0.71.1-A301F6B2 darwin/amd64 BuildDate: 2020-05-25T09:09:07Z

Does this issue reproduce with the latest release?

Yes

What is the issue?

Nesting a markdown shortcode within a markdown shortcode can in some cases cause a codefence to render some of its contents as markdown.

This doesn't appear to happen if there is just one shortcode surrounding the markdown. e.g removing {{< first >}} and {{< /first >}} from the test example below.

Appears to happen with both methods of specifying markdown:

• {{% ... %}} OR
• {{< ... >}} with .Inner | markdownify

Screenshot of issue

Test

Bit of a messy test but you get the point I added this to hugolib/shortcode_test.go and the line saying # This declares a view into HAProxy statistics, on port 3833 gets rendered as a h1 even though its inside a code block.

func TestNestedShortCodeWithCodeFence(t *testing.T) {
	t.Parallel()
	c := qt.New(t)

	builder := newTestSitesBuilder(t)
	builder.WithConfigFile("yaml",
		`
pygmentsuseclasses: true
pygmentsCodeFences: true

markup:
  goldmark:
    renderer:
      unsafe: true
  highlight:
    guessSyntax: true
`)

	builder.WithContent("page.md", `---
title: "Hugo Rocks!"
---

{{< first >}}

# This is first markdown

{{< second >}}

# This is second markdown

`+"```"+`
# Basic configuration
timeout client 5s
timeout server 5s
timeout connect 5s

# This declares a view into HAProxy statistics, on port 3833
mode http
stats enable
`+"```"+`
{{< /second >}}
{{< /first >}}
`)
	builder.WithTemplatesAdded("layouts/shortcodes/first.html", `<div>{{- .Inner | markdownify -}}</div>`)
	builder.WithTemplatesAdded("layouts/shortcodes/second.html", `<div>{{ .Inner | markdownify }}</div>`)
	builder.Build(BuildCfg{})

	s := builder.H.Sites[0]
	c.Assert(len(s.RegularPages()), qt.Equals, 1)

	builder.AssertFileContent("public/page/index.html", `<div><h1 id="this-is-first-markdown">This is first markdown</h1>
<div><h1 id="this-is-second-markdown">This is second markdown</h1>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"># Basic configuration
timeout client 5s
timeout server 5s
timeout connect 5s

# This declares a view into HAProxy statistics, on port 3833
mode http
stats enable
</code></pre></div></div></p>
</div>`)
}

[moorereason]

Confirmed. The test content can be reduced to:

---
title: "Hugo Rocks!"
---
{{< first >}}
# This is first markdown
{{< second >}}
# This is second markdown
```
# COMMENT 1

# COMMENT 2
```
{{< /second >}}
{{< /first >}}

The key is to have a blank line in the code section.

[davidejones]

Thank you, much more concise! and yes we actually removed the spacing as a workaround to stop the issue.

[satotake]

s1 := `{{< first >}}
# This is first markdown

{{< second >}}

# This is second markdown

`+"```"+`
# Comment 1

# Comment 2
`+"```"+`
{{< /second >}}
{{< /first >}}
`

After inner markdownify, s1 is transformed like this as we expect

# This is first markdown

<div><h1 id="this-is-second-markdown">This is second markdown</h1>
<pre><code># Comment 1

# Comment 2
</code></pre></div>

According to commonmark, the above results in <h1> inside <pre>

Although this is a little surprise, it might not be a bug.

@davidejones Is it fine for you to remove inner markdownify? I mean

	builder.WithTemplatesAdded("layouts/shortcodes/first.html", `<div>{{- .Inner | markdownify -}}</div>`)
	builder.WithTemplatesAdded("layouts/shortcodes/second.html", `<div>{{ .Inner }}</div>`)

This is the easiest way to fix this, I think.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.