Github workflow commands (::error, ::warning, etc) from a task not picked up

[ericmatthys]

What version of Turborepo are you using?

1.2.16

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Describe the Bug

Github Actions may rely on annotations from reporters for associating errors with a particular file. This is used by the Jest github-actions reporter. I'm having trouble getting the reporter to work in a monorepo with Turborepo, I think because the message is printed with the package + task prefix.

https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-error-message

The output from Turborepo:

core:test: ::error file=/home/runner/work/website/website/packages/core/utils/object/pick.spec.ts,line=9,title=pick › should ignore undefined properties::expect(received).toEqual(expected) // deep equality%0A%0A- Expected  - 1%0A+ Received  + 2%0A%0A  Object {%0A-   "a": 2,%0A+   "a": 1,%0A+   "b": undefined,%0A  }%0A%0A    at Object.toEqual (utils/object/pick.spec.ts:9:52)%0A    at TestScheduler.scheduleTests (../../node_modules/.pnpm/@[email protected]/node_modules/@jest/core/build/TestScheduler.js:317:13)%0A    at runJest (../../node_modules/.pnpm/@[email protected]/node_modules/@jest/core/build/runJest.js:407:19)%0A    at _run10000 (../../node_modules/.pnpm/@[email protected]/node_modules/@jest/core/build/cli/index.js:339:7)%0A    at runCLI (../../node_modules/.pnpm/@[email protected]/node_modules/@jest/core/build/cli/index.js:190:3)

Expected Behavior

I'd expect the Jest github-actions reporter's annotations to be picked up by Github automatically. Perhaps this style of annotation should avoid the prefix and be printed directly, if that is actually the issue here.

To Reproduce

Using Jest 28.1.1, add reporters: ['default', 'github-actions'] to the Jest config file to enable the reporter. Add a failing test and trigger a workflow that runs the Jest tests within a Turborepo task.

package.json

"test": "cross-env FORCE_COLOR=1 turbo run test"

packages/core/package.json

"test": "cross-env TZ=UTC jest"

.github/workflows/test.yml

name: Run tests

on:
  pull_request:
  push:
    branches:
      - staging

env:
  TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
  TURBO_TEAM: ${{ secrets.TURBO_TEAM }}

jobs:
  test:
    name: Run tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: pnpm/[email protected]
        with:
          version: 7.1.7

      - uses: actions/setup-node@v3
        with:
          node-version: '16'
          cache: 'pnpm'

      - name: Install packages
        run: pnpm install --frozen-lockfile

      - name: Run unit tests
        run: pnpm run test

[nathanhammond]

Tagging in #1349

[zomars]

We're having the same issue. We have a turbo run type-check that runs tsc on each package and the GitHub annotations are being suppressed due turbo log handling/grouping/formatting. Running tsc directly on each package respects the annotations, but running them from turbo doesn't.

Here's the pattern that setup-node uses internally.

[weyert]

Is this similar to Github Checks to add warning markers to code? https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api

[zomars]

@weyert I'm talking more about these kind of annotations:

[ddoice]

Not perfect but in the meantime we have created a small script to wrap our jest and playwright scripts to capture these errors and remove the package + task prefix.

.github/error_trap.sh

#!/bin/bash
$@ |& tee output.txt
cat output.txt | grep " ::error " | sed -e "s/^.*:.*:\s::error/::error/"
failed=$(cat output.txt | grep " ::error " | wc -l)

if [ $failed -gt "0" ]
then
  exit 1
fi
exit 0

Then change your jobs steps, IE:

# - run: yarn run e2e
- run: >
    bash -e ./.github/error_trap.sh
    yarn run e2e

[dominikg]

playwright also has a github reporter that uses ::notice and there is also error reporting that parses log lines. see https://github.com/sveltejs/kit/actions/runs/3465678222 for an example where the run overview has additional information available that would otherwise be hidden in a flurry of logs behind matrix jobs.

github has another feature that allows you to group log lines: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#grouping-log-lines

so if turbo detected the github action env and then used that instead of prefixing each line, we'd get all the niceties without losing the task association. #1349 discussed difficulties with replays and caching, not sure how exactly but capturing the logs in a raw format per task and log replay then using process.env.CI && process.env.GITHUB_ACTIONS to format logs accordingly would be great.

[zomars]

Would adding a way to remove the turbo package prefixes in CI help here?

[gsoltis]

@zomars Yes, absolutely. I believe there is a PR for it that would be a good starting point, although it may have stalled.

#3439 would be great to pick up or use as inspiration.

[kpervin]

This still seems to be an issue as of August 2023. When attempting to use the github-actions reporter in Jest, annotations are not correctly added when run through Turborepo, but run fine when run standalone.

[mehulkar]

You can use --log-prefix=none to remove the prefix in CI. I have not tested to see if GH Actions annotations from jest still work however.

[mehulkar]

I believe this is working now, even without --log-prefix. I tried this out in another project and GH annotations are working correctly.