snap: Failure to resolve media type with resources.GetRemote

[arjunrbery]

What version of Hugo are you using (hugo version)?

$ hugo version
hugo v0.119.0-b84644c008e0dc2c4b67bd69cccf87a41a03937e+extended linux/amd64 BuildDate=2023-09-24T15:20:17Z VendorInfo=snap:0.119.0

Does this issue reproduce with the latest release?

yes

Steps to reproduce

1. Create a new repo
2. Add config.yaml with the following contents

# config.yaml
disableKinds:
- sitemap
- taxonomy
- term
outputs:
  home:
  - html

3. Add layouts\index.html with the following contents

{{/* prebuild/layouts/index.html */}}
<ul>
{{ with resources.GetRemote "https://blog.arb.dev/feed/?t=1698195621" | transform.Unmarshal }}
  {{ range .channel.item }}
    {{ $timestamp := time .pubDate }}
    {{ $link := .link | plainify | htmlUnescape }}
    {{ $postID := replace $link "https://blog.arb.dev/" "" }}
    {{ $postID := replace $postID "/" "-" }}
    {{/* 1. */}} {{ $string := printf "<h1>%s</h1>%s" .title .description }} 
    {{/* 2. */}} {{ $filename := printf "post/%s/%s.md" (urlize $timestamp.Year) (urlize $postID) }} 
    {{/* 3. */}} {{ $resource := resources.FromString $filename $string }} 
    {{/* 4. */}} {{ $file := $resource.RelPermalink }}
    <li><a href="{{$filename}}">{{$filename}}</a></li>
  {{ end }}
{{ end }}
</ul>

4. Run hugo server and you will see a successfully generated page with links to all of files that have been generated

5. Run hugo and no child files will be generated and the resulting index.html page will be blank

Additional info: If you print the output of resources.GetRemote "https://proton.me/blog/feed" | transform.Unmarshal on build it generates the following error.

map[error calling resources.GetRemote:failed to resolve media type for remote resource "https://blog.arb.dev/feed/?t=1698195621"] 

Please note, I'm using https://blog.arb.dev/feed/?t=1698195621 as a stand in for the this example.

[jmooring]

I am unable to reproduce the problem as described.

git clone --single-branch -b hugo-github-issue-11601 https://github.com/jmooring/hugo-testing hugo-github-issue-11601
cd hugo-github-issue-11601
rm -rf resources/ public/ && hugo

Then look at the contents of the public directory:

tree public

Then examine the published home page:

grep -A6 "<ul>" public/index.html 

Unless I'm missing something, this seems like a more of a support issue than a bug, so it would be best to continue the conversation on the forum.

[jmooring]

I just noticed that you installed Hugo using the snap package, and now I can recreate the problem. Remove the snap package and install using a prebuilt binary.

[arjunrbery]

Thanks for the prompt response. The prebuilt binary resolved the issue.

[jmooring]

EDIT 2025-08-30T09:04:13-07:00

The original resource URL is no longer functional. Use https://www.rassoc.com/gregr/weblog/feed/ for testing.

---

This is a bug, a limitation of the snap package, so I'm going to re-open. I don't see a short term fix, but it https://github.com/gohugoio/hugo/issues/11126 would fix the problem. I can also reproduce the problem with a Docker container. In either case, snap or docker, I can retrieve a remote JSON file without error.

The media type returned by the remote server when retrieving https://blog.arb.dev/feed/?t=1698195621 is:

Content-Type: application/rss+xml; charset=utf-8

Go's built-in media types do not include the above. See https://go.dev/src/mime/type.go. So, Go's mime package tries to look at the local system media type db (e.g., on Ubuntu /etc/mime.types, /usr/share/mime/globs2, etc.), but neither the snap (strictly confined) nor the docker image have access to those files.

With the snap package, maybe we can get a plug to the required files... need to test. Or go down the #11126 path, which I'd rather avoid for now.

---

No problems when running hugo server with either the snap or docker container. When running hugo server we load an internal list of media list types, but we don't do that when running hugo.

---

As expected, testing with https://daringfireball.net/thetalkshow/rss works fine with the snap and docker image. Their server returns:

Content-Type: text/xml; charset=utf-8

And that media type is one of Go's built-in media types.

---

I can get it working with docker if I install shared-mime-info when building the image, but that approach doesn't work with the snap; neither does creating a system-files plug to access /etc/mime.types. I think we have to wait for https://github.com/gohugoio/hugo/issues/11126.

[arjunrbery]

@jmooring I'll also leave that page up for you all to use for testing. Once I see this close, I'll take down that website.

[jmooring]

@bep, My memory is a bit fuzzy on this. We do this when running hugo server:

https://github.com/gohugoio/hugo/blob/c4a530f104f2b0d1dd82211b77b603a07db7c181/hugolib/site.go#L398-L406

Is there a downside to doing this with resources.GetRemote?

[arjunrbery]

Hi @jmooring,

I wanted to see if I can pull down that temporary site since the ticket is now unassigned.

Thanks

[jmooring]

@arjunrbery Yes, thanks, please proceed.

[bep]

Is there a downside to doing this with resources.GetRemote?

I don't see any downsides.

[toby3d]

I reproduce the same behavior in the built binary 0.129.0-1 installed via pacman (extra/hugo) on Arch Linux. Similar error when trying to request Mastodon widget via public API, Content-Type which is application/json; charset=utf-8:

{{- with resources.GetRemote $url (dict "headers" (dict "Accept" "application/json")) -}}
  {{ with .Err }}
    {{ errorf "error during fetch remote resource: %s" . }}
  {{ else }}
    {{ $data = . | transform.Unmarshal }}
    <figure>
      {{- $data.html | safeHTML -}}
    </figure>
  {{ end }}
{{- else -}}
  {{ errorf "cannot get remote resource %q" $url }}
{{- end -}}

ERROR error during fetch remote resource: error calling resources.GetRemote: failed to resolve media type for remote resource "https://mstdn.io/api/oembed?maxheight=0&maxwidth=1024&url=https%3A%2F%2Fmstdn.io%2F%40toby3d%2F110780380038894309"

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. If this is a feature request, and you feel that it is still relevant and valuable, please tell us why. This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[jmooring]

With the snap package I can get this to work doing both of the following:

1. Adding the served media type to the security allowlist:

   [security]
     [security.http]
       mediaTypes = ['^application/rss\+xml; charset=UTF-8$']
   

2. Passing the resource's .Content to transform.Unmarshal instead of passing the resource itself:

   {{ with .Content | transform.Unmarshal }}
   

But we shouldn't need to do this. See https://github.com/gohugoio/hugo/issues/11601#issuecomment-1781358500 above.