Update Wiki task : error "Argument list too long"

[Styxxy]

Azure DevOps Extensions

WIKI Updater

Platform

Azure DevOps Services

Azure DevOps Server (TFS) Version

N/A

Extension Version

1.37.2

Describe the bug

Updating a wiki page results in error "Argument list too long" on ubuntu-latest MSHosted Agents.

It used to work; last build that uploaded was 17 June 2022. Now (18 July) the task fails.

Repo Steps

Using following template:

parameters:
- name: environment
  type: string
- name: targetWiki
  type: string
  default: 'dev.azure.com/<org>/<project>/_git/<project>.wiki'
- name: targetWikiFolder
  type: string
  default: 'Release-Notes'

jobs:
- job: ReleaseNotesJob${{ parameters.environment }}
  displayName: 'Release Notes ${{ parameters.environment }}'
  condition: succeeded()
  pool:
    vmImage: 'ubuntu-latest'
  steps:
  - task: XplatGenerateReleaseNotes@3
    displayName: 'Calculate Release Notes'
    inputs:
      outputfile: '$(System.DefaultWorkingDirectory)\releasenotes.md'
      outputVariableName: 'OutputText'
      templateLocation: 'File'
      templatefile: '.azure-pipelines/releasenotes/wiki-template.md'
      customHandlebarsExtensionFolder: '$(Build.SourcesDirectory)/.azure-pipelines/releasenotes'
      customHandlebarsExtensionFile: 'handlebars-extension.js'
      checkStage: true
      dumpPayloadToConsole: false
      dumpPayloadToFile: false
      replaceFile: false
      appendToFile: true
      getParentsAndChildren: true
      getAllParents: true
      getIndirectPullRequests: false
      stopOnRedeploy: true
      stopOnError: true
  - task: WikiUpdaterTask@1
    displayName: 'Update Wiki ${{ parameters.environment }}'
    inputs:
      repo: ${{ parameters.targetWiki }}
      filename: '${{ parameters.targetWikiFolder }}/${{ parameters.environment }}.md'
      replaceFile: false
      appendToFile: false
      dataIsFile: true        
      sourceFile: '$(System.DefaultWorkingDirectory)\releasenotes.md'
      message: 'Push release notes for pipeline $(Build.BuildNumber)'
      tagRepo: false
      gitname: 'azure pipelines'
      gitemail: 'builduserazdo'
      useAgentToken: true

Expected Behavior

Should upload release notes. Used to work.

Logging Information

==============================================================================
Task         : Git based WIKI Single File Updater from Black Marble
Description  : A tools to update or create a file in a WIKI based in a Git repo 
Version      : 1.37.2
Author       : Richard Fennell
Help         : Version: 1.37.2. [More Information](https://github.com/rfennell/vNextBuild/wiki/Wiki-Updater-Task/)
==============================================================================
##[debug]Using node path: /home/vsts/agents/2.204.0/externals/node10/bin/node
##[error]Argument list too long
##[debug]System.ComponentModel.Win32Exception (7): Argument list too long
   at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.NodeHandler.RunAsync()
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync()
   at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
Finishing: Update Wiki

[rfennell]

This has been reported before in #1024 - we did not get a resolution or repo, the issue just went away.

I root cause I think, as the error suggest, is the environment variable using used to pass the parameter see https://stackoverflow.com/questions/28865473/setting-environment-variable-to-a-large-value-argument-list-too-long for a discussion.

However, I can't see why the parameter you have set on the task should be an issue, the are file paths - unless they are too long?

As I suggested in #1024 do you get the same issues on the Windows based agents?

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[phantran]

I have the same issue, can't find any solutions to this

[rfennell]

The error message show is prior to any of my code being run. For normal operations I expect to see

022-09-26T15:00:54.0932970Z ##[debug]Evaluating condition for step: 'Git based WIKI Updater - Azure DevOps with Creds update .order file' 2022-09-26T15:00:54.0935697Z ##[debug]Evaluating: SucceededNode() 2022-09-26T15:00:54.0936680Z ##[debug]Evaluating SucceededNode: 2022-09-26T15:00:54.0938178Z ##[debug]=> True 2022-09-26T15:00:54.0939079Z ##[debug]Result: True 2022-09-26T15:00:54.0940007Z ##[section]Starting: Git based WIKI Updater - Azure DevOps with Creds update .order file 2022-09-26T15:00:54.1087927Z ============================================================================== 2022-09-26T15:00:54.1088816Z Task : Git based WIKI Single File Updater from Black Marble 2022-09-26T15:00:54.1089561Z Description : A tools to update or create a file in a WIKI based in a Git repo 2022-09-26T15:00:54.1090177Z Version : 1.43.6 2022-09-26T15:00:54.1090669Z Author : Richard Fennell 2022-09-26T15:00:54.1091433Z Help : Version: 1.43.6. More Information 2022-09-26T15:00:54.1092180Z ============================================================================== 2022-09-26T15:00:54.1193877Z ##[debug]Using node path: C:\agents\2.210.1\externals\node10\bin\node.exe 2022-09-26T15:00:54.2833285Z ##[debug]agent.TempDirectory=D:\a_temp

lots of lines of autogenerated logging prior to my code being run

2022-09-26T15:00:54.3308178Z ##[debug]injecttoc=False

and this is the first line of logging I show. the only code of mine run is to read the parameters.

2022-09-26T15:00:54.3313608Z Variable: Repo [dev.azure.com/richardfennell/Git%20project/_git/Git-project.wiki]

So whatever the issue is, it is happening before my task runs. All I can assume is that there is some overly long parameter being passed in.

[Wintekso]

Hello,

I found that having output variables set on previous tasks was the cause for this.

We were using the XplatGenerateReleaseNotes large template, all output variables are passed to new tasks.

Hope this helps

[rfennell]

Of course good spot @Wintekso . It is very easy to get the generated set of release notes that are too big to store as an output variable, which remember is just an environment variable and gave a maximum size that varies based on the OS

So yes, disabling the use of output variables is a very good idea