CI pipelines hang indefinitely on `salus -h`

[lee0c]

In two sample CI pipelines, one using Azure DevOps and one using GitHub Actions (both linux agents), I've so far found that salus -h hangs indefinitely until the task/step is canceled by the pipeline or a user (up to 60 min on AzDO, 20 min on GH - just hitting standard pipeline timeouts), but a properly parameter-supplied salus generate command can run fine.

Both systems have logging output of:

scripts/salus-install.sh: line 22:  1660 Killed                  salus -h

and report exit code 137.

While I've had unrelated (I think) issues running Salus locally, running salus -h in an Azure Cloud Shell session works fine and returns promptly, so the error is not solely with the command.

[ByAgenT]

Was able to repro hang in Docker for salus -h and just salus in container as well:

FROM ubuntu:20.04

RUN apt update && apt install curl -y

RUN curl -Lo salus https://github.com/microsoft/sbom-tool/releases/download/v0.1.2/sbom-tool-linux-x64 \
 && chmod +x salus \
 && mv salus /usr/local/bin

CMD ["salus", "-h"]

Generation flow though works fine and completes:

FROM ubuntu:20.04

WORKDIR /sbom

RUN apt update && apt install curl -y

RUN curl -Lo salus https://github.com/microsoft/sbom-tool/releases/download/v0.1.2/sbom-tool-linux-x64 \
 && chmod +x salus \
 && mv salus /usr/local/bin

RUN mkdir test

CMD ["salus", "Generate", "-b", "./test", "-bc", "./test", "-pn", "Test", "-pv", "0.0.1", "-nsb", "http://test.sbom", "-V", "Verbose"]

What's curious is that if I bash into container via docker exec I can run salus -h just fine. Could be something wrong w/ library that outputs help to stdout. Needs more investigation.

[bureado]

Can repro in GHA, in Cloud Shell and WSL2 when redirecting salus -h output to a file and using a limited environment: env -i HOME=/home/bureado /usr/local/bin/salus -h > xxa

[bureado]

There appears to be a compound scenario where STDOUT for salus isn't a terminal (but a pipe or a file, for example) and TERM isn't set, which seems concomitant in GHA, and likely ADO. I don't know enough about .NET Core to track down which NuGet dependency(ies) are involved.

As a workaround set TERM to something. For example, TERM=linux.

Update: this appears to be an issue with PowerArgs; I was able to reproduce this with one of the samples.

[ByAgenT]

@bureado thanks a lot for your investigation and for providing workaround! I would expect the same scenario applies to Docker containers as well.

Also it seems that main scenario of SBOM generation isn't blocked because of this issue and tool should produce SBOM in both GHA/ADO with valid arguments.

@aasim FYI. We probably need to check if that's fixable in our code and submit an issue/fix to PowerArgs repo.

[bureado]

I would also try #47. Using 6.0 to build PowerArgs HEAD I get samples that don't hang. I just don't know if that's because of newer code, or because of 6.0.

[Brend-Smits]

There appears to be a compound scenario where STDOUT for salus isn't a terminal (but a pipe or a file, for example) and TERM isn't set, which seems concomitant in GHA, and likely ADO. I don't know enough about .NET Core to track down which NuGet dependency(ies) are involved.

As a workaround set TERM to something. For example, TERM=linux.

Update: this appears to be an issue with PowerArgs; I was able to reproduce this with one of the samples.

This workaround does not seem to work for me when using the GitHub cloud runners (ubuntu-22). I'm trying with:

  test-local:
    runs-on: ubuntu-latest
    env:
      TERM: linux
    steps:
      - uses: actions/checkout@v2
      - run: |
          TERM=linux sbom-tool -h

Tried by setting the environment variable in the step level, as well as when executing the command. In both scenarios, it will hang indefinitely. Am I missing something?