multilingual: Shared resource discarded if same bundle does not exist in default content language

[jmooring]

Similar to #12079, but there's no ambiguity in this case: we should not discard the page resource.

Below, en is the default content language.

content/
└── s1/
    ├── p1/
    │   ├── a.txt
    │   └── index.de.md  <-- non-default content language, no matching bundle in english
    ├── _index.de.md
    └── _index.en.md

Expected (v0.122.0)

public/
├── de/
│   ├── s1/
│   │   ├── p1/
│   │   │   ├── a.txt      <-- this is missing in v0.123.1 see below
│   │   │   └── index.html
│   │   └── index.html
│   └── index.html
├── en/
│   ├── s1/
│   │   └── index.html
│   └── index.html
└── index.html

Actual (v0.123.1)

public/
├── de/
│   ├── s1/
│   │   ├── p1/
│   │   │   └── index.html
│   │   └── index.html
│   └── index.html
├── en/
│   ├── s1/
│   │   └── index.html
│   └── index.html
└── index.html

failing test case

func TestFoo(t *testing.T) {
	t.Parallel()

	files := `
-- hugo.toml --
disableKinds = ['rss','sitemap','taxonomy','term']
defaultContentLanguage = 'en'
defaultContentLanguageInSubdir = true
[languages.en]
[languages.de]
-- layouts/_default/single.html --
{{ .RelPermalink }}
-- layouts/_default/list.html --
{{ .RelPermalink }}
-- content/s1/_index.de.md --
---
title: s1 (de)
---
-- content/s1/_index.en.md --
---
title: s1 (en)
---
-- content/s1/p1/index.de.md --
---
title: s1/p1 (de)
---
-- content/s1/p1/a.txt --
irrelevant
`

	b := hugolib.Test(t, files)

	b.AssertFileExists("public/de/s1/index.html", true)
	b.AssertFileExists("public/de/s1/p1/index.html", true)
	b.AssertFileExists("public/de/s1/p1/a.txt", true) // failing test

	b.AssertFileExists("public/en/s1/index.html", true)
	b.AssertFileExists("public/en/s1/p1/index.html", false)
	b.AssertFileExists("public/en/s1/p1/a.txt", false)
}

References:

• https://discourse.gohugo.io/t/resources-getmatch-issue-in-0-123/48619
• https://github.com/gohugoio/hugo/issues/12189#issuecomment-1973705053 (item 5)

[bep]

content/
└── s1/
    ├── p1/
    │   ├── a.txt <--- this is by definition an English resource
    │   └── index.de.md  <-- non-default content language, no matching bundle in english
    ├── _index.de.md
    └── _index.en.md

I'm sure about this.

• In 0.122.0 we created missing bundle headers for "dangling resources"
• The simplification in 0.123.0 was made partly to prepare for other content data sources, and we don't want to have to "look at the whole thing" to determine where a piece of content belongs.
• a.txt is English. I don't want to open up to saying that it may change language depending on some other file presence.

[nekr0z]

• a.txt is English. I don't want to open up to saying that it may change language depending on some other file presence.

For clarity, not questioning the decision itself: are you explicitly and definitely saying a.json and a.jpg are English, too, and can't be considered to be in other languages no matter what?

[jmooring]

are you explicitly and definitely saying a.json and a.jpg are English

No. They are tied to the default content language as specified in your site configuration, falling back to en if the default content language is not specified in site configuration.

[nekr0z]

As I see it, this is a fundamental divergence from the mutlilingual logic in Hugo.

The basic multilingual logic for missing translations is, as documented: "If a string does not have a translation for the current language, Hugo will use the value from the default language. If no default value is set, an empty string will be shown." Effectively: _string in the language_ | default _string in default language_ | default "".

Up until 0.122, the same logic applied to resources. Here, this logic is broken.

While the page in the default language does not exist, the resource (in the default language) does exist, is explicitly called (from the page in another language), and should be used as a fallback, instead of falling right through to the empty value.

[markdumay]

To add my two cents, I noticed this behavior applies to image resources as well. My site - available in both English (default) and Dutch - contains several pages that use page resources. This is done mainly to improve the site's asset management (these "local" images are very specific to a blog post, and are not reused elsewhere).

Consider the following example:

.
└── content
    └── blog
        └── my-first-post
            ├── img
            │   └── image1.jpg
            └── index.nl.md

In this setup, image1.jpg is not available as page resource to index.nl.md. When I add index.en.md, the image does become available to both translated pages. IMHO, these (image) resources should be available regardless if a page in the default language is available.

[brmassa]

Here too. It breaks my site!!!

My blog has 2 languages and most posts are in only one of them. If there is no default language post, all the images are ignored.

{{ .Resources }} is returning [] if no default index.md is present

[bep]

I will take one more look at this later today and see if I can make "an exception" for content files (which is what we have today), but some goals with this new setup was to

• Make it easy to insert/update content without having to know too much about the surrounding context.
• Make that not dependent on any order so we could do it concurrently.

[CXwudi]

I would like to share my circumstance as well. My folder structure is as following:

.
└── content
    └── posts
        └── my-first-post
            ├── img.png
            ├── index.en.md
            └── index.zh-cn.md

The default language is set to English.

Using the figure shortcode, I can see that img.png is properly showing in index.en.md but not index.zh-cn.md,

[jmooring]

@CXwudi Your problem is not related to this issue. Please create a new issue.

[CXwudi]

Created the issue at https://github.com/gohugoio/hugo/issues/12244

(I modified my comment to avoid ambiguity)

[brmassa]

0.125.3 the problem persists... @bep, @jmooring, reopen it please.

{{ .Resources.ByType "images" }} will return [] if there is only a non-default page.

[jmooring]

@brmassa

reopen it please

This issue is open. Are you referring to something else?

[brmassa]

My bad. The CXwudi comment fooled me.