Downloading private modules not working with Hugo v0.91.0 or higher

[dvdksn]

What version of Hugo are you using (hugo version)?

$ hugo version
hugo v0.91.2-1798BD3F+extended windows/amd64 BuildDate=2021-12-23T15:33:34Z VendorInfo=gohugoio

Does this issue reproduce with the latest release?

Yes

Observation

With Hugo v0.90.1 and earlier versions, I'm able to hugo mod get -u modules from a private registry from an on-prem gitlab instance. But with v0.91 and later, the hugo mod get -u command just hangs.

Steps to reproduce:

> hugo new site my-site
> cd .\my-site\
> Add-Content .\config.toml "
>>
>> [[module.imports]]
>> path = 'private.gitlab.instance/module/path'"
> cat .\config.toml
baseURL = 'http://example.org/'
languageCode = 'en-us'
title = 'My New Hugo Site'

[[module.imports]]
path = 'private.gitlab.instance/module/path'
> hugo version
hugo v0.91.0-D1DC0E9A+extended windows/amd64 BuildDate=2021-12-17T09:50:20Z VendorInfo=gohugoio
> hugo mod get -u
hugo: downloading modules …

Then it proceeds to hang on downloading modules. With a different Hugo version:

> hugo version
hugo v0.90.1-48907889+extended windows/amd64 BuildDate=2021-12-10T10:56:41Z VendorInfo=gohugoio
> hugo mod get -u
hugo: downloading modules …
go get: added private.gitlab.instance/module/path v0.0.0-20211230124009-6e6c34460737
hugo: collected modules in 2667 ms

[bep]

Can you check if setting HUGO_SECURITY_EXEC_OSENV=".*" OS env makes it work for you?

[dvdksn]

Can you check if setting HUGO_SECURITY_EXEC_OSENV=".*" OS env makes it work for you?

This worked! Should I run with this configuration, or set something more specific?

[khayyamsaleem]

On my machine, without this environment variable, it started asking for my Username via an interactive prompt, despite having the .gitconfig replacement configurations in place to use SSH instead of https. Would fail on any hugo mod commands. After adding the env var as @bep mentioned, it was able to fetch the private modules as usual.

[UtkarshVerma]

I have also faced the same issue on the same Hugo version as OP. bep's suggestion works for me as well. HUGO_SECURITY_EXEC_OSENV=".*"

[jmooring]

@dvdksn @khayyamsaleem @UtkarshVerma

If you have some time, see if you can determine which environment variable(s) are required, starting with the usual suspects (HOME, LANG, USER, USERNAME, PWD, GOPROXY, etc.). Keep in mind that you might have to add more than one to security.exec.osEnv.

Reference: https://gohugo.io/about/security-model/#security-policy

[nathlaroche]

We're also having the same issue here, and Bep's suggestion works as well. Thanks for the workaround!

[jmooring]

@nathlaroche Please see my previous comment, and help narrow this down if you can. Thanks.

[UtkarshVerma]

@jmooring I tried the above-mentioned env-vars, with no difference. What else should I test?

[jmooring]

@UtkarshVerma Look at the env vars that are set on your system. With Linux that's the env command.

[UtkarshVerma]

@jmooring I investigated this bug. I narrowed down the requirements to the following:

export HUGO_SECURITY_EXEC_OSENV="(?i)^(PATH|XDG_CONFIG_HOME|SSH_AUTH_SOCK)$"

I can understand SSH_AUTH_SOCK being needed by go for authentication purposes, but XDG_CONFIG_HOME has me clueless.

[dvdksn]

On my end, I managed to narrow it down to these variables:

[security]
[security.exec]
osEnv = ['(?i)^(Path|USERPROFILE)$']

Thanks everyone for your support, I'll close this ticket as I consider the problem (user error) resolved.

[UtkarshVerma]

Could you please not close this issue until this is fixed upstream.

By that, I mean until acceptable defaults have been set for the config.

[acalvo]

FYI just adding HOME to de default osEnv worked in my case (CI environment).

[jmooring]

@UtkarshVerma I'm pretty sure the XDG_CONFIG_HOME dependency is related to git, or to how you have configured git.

See https://git-scm.com/docs/git-config#FILES

[UtkarshVerma]

@UtkarshVerma I'm pretty sure the XDG_CONFIG_HOME dependency is related to git, or to how you have configured git.

See https://git-scm.com/docs/git-config#FILES

Yes, I do keep my git config in XDG_CONFIG_HOME.

[jmooring]

Summarizing what we know so far...

The current (v0.92.0) default is:

[security.exec]
  osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

Based on the comments above, it looks like this would need to be:

[security.exec]
  osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM|HOME|SSH_AUTH_SOCK|USERPROFILE|XDG_CONFIG_HOME)$']

[UtkarshVerma]

I tracked down the env vars which were exactly required in my case - $XDG_CONFIG_HOME and $SSH_AUTH_SOCK. Both of these are as @jmooring suggested. My Git config is placed in $XDG_CONFIG_HOME so hugo could only switch to SSH based authentication for cloning modules if this rule would be applied, which it wasn't:

[url "ssh://[email protected]/"]
	insteadOf = https://github.com/

After that, git subprocess should also be able to use the SSH socket, hence the need for $SSH_AUTH_SOCK. I hope these variables are added upstream.

The final regex with which I tested was:

(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|XDG_CONFIG_HOME|SSH_AUTH_SOCK)$

[UtkarshVerma]

Please let me know if you need additional information on this issue. https://github.com/gohugoio/hugo/issues/9333#issuecomment-1356306256 has been working for me consistently. It would be great to have this fixed upstream.

[giuliohome]

This is relevant for hugo modules from private (under adfs sso) gitlab repositories, when GOPRIVATE env is needed. Eventually I had to set osEnv = ['.*'] in the security section of toml configuration! A better error message hint would help.

[security]
  exec.osEnv =".*"

[rhcarvalho]

I was having problems with Hugo Modules and a private git repository as well.

I have a insteadOf configuration in my .gitconfig, but that is ignored under hugo mod tidy, etc.

The fix is to add HOME to security.exec.osEnv -- I see it has been added and then removed in previous versions.

Is there a problem adding it there by default again? Right now, without HOME, it is harder to use Hugo Modules with private repositories.

[rhcarvalho]

Upon further investigation, and as cited in previous comments, I also needed SSH_AUTH_SOCK to make fetching private repositories over SSH work.

So my hugo.toml became:

# other configs...

[security]
  [security.exec]
    osEnv = '(?i)^((HTTPS?|NO)_PROXY|PATH(EXT)?|APPDATA|TE?MP|TERM|GO\w+|HOME|SSH_AUTH_SOCK)$'

A consequence of putting this in our project config is that if the default ever changes, we won't automatically pick up the changes (could be seen as positive or negative).

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.