cc-test-reporter still broken with SimpleCov

[stepchud]

I have a dependabot issue open on my repo to upgrade SimpleCov from 0.17.1 => 0.22.0, which is failing on CircleCI. The error message I am experiencing is the same issue that was reported in [#413, #418], however none of the suggested solutions are working for me.

I have tried every combination of ways to configure environment in CirceCI, and I have confirmed with puts statements that the CC_TEST_REPORTER_ID is available and correct during the build.

1. it was originally already configured in the Project Settings on the CircleCI build
2. using the :environment key on the rspec job
3. using the :env key on the step when rspec is executed
4. using export CC_TEST_REPORTER_ID=... as the first line of the command array
5. applying it directly to the command CC_TEST_REPORTER_ID=... bundle exec rspec

I added the gem to my bundle and explicitly set the formatter:

require 'simplecov_json_formatter'
SimpleCov.formatter = SimpleCov::Formatter::JSONFormatter

I can also see the formatter for SimpleCov is correct via puts (SimpleCov::Formatter::JSONFormatter). I also tried using the simplecov-json gem to no avail because it is mentioned on some docs.

We have been using a gem called rspec_junit_formatter and I tried removing that in case it was causing issues, just trying to remove variables and simplify what might be wrong.

Here is the step that's failing in CircleCI:

      - run:
          name: Run Tests
          env:
            CC_TEST_REPORTER_ID: a561...0703
          command: |
            mkdir /tmp/test-results
            ./cc-test-reporter before-build

            # Run Rspec tests
            echo "Run Rspec tests"
            CC_TEST_REPORTER_ID=a561...0703 bundle exec rspec

            echo "cc-test-reporter format-coverage"
            CC_TEST_REPORTER_ID=a561...0703 ./cc-test-reporter format-coverage \
              --output coverage/codeclimate.rspec.$CIRCLE_NODE_INDEX.3.2.4.json

Just to be clear I am using the actual key, not the ellipses present in this example. CodeClimate states in their documentation that they don't consider CC_TEST_REPORTER_ID to be a secret that needs to be protected but I didn't feel comfortable posting it here anyways. The error message output after the bundle exec rspec command:

Error: json: cannot unmarshal object into Go struct field resultSet.coverage of type []formatters.NullInt

I was initially able to reproduce the error locally. I downloaded cc-test-reporter binary for arm64 and have it on my PATH. Exporting CC_TEST_REPORTER_ID env var worked to fix the output locally, but not on CircleCI. I have tried all of the command line options to cc-test-reporter both locally and on Circle but they're no help. I've spent 2 days trying to get this to work. My last resort will be to try to debug the CircleCI run via SSH since the puts are not helping.

CC: @fede-moya if you are still offering personal assistance I will try to email you directly.