Screenshots have variable color tint

[zommerfelds]

Hi, not sure if this is related to this package or not but I couldn't find anything on the internet so far about this issue.

I'm running screenshot comparison tests on GitHub Actions, but the screencap output has a variable white tint depending on the run.

Do you have any ideas why this could be? Here are the screenshots captured by adb exec-out screencap -p > blah.png. I have a loop where I keep retrying until it matches.

Here's my setup:

      - name: AVD cache
        uses: actions/cache@v2
        id: avd-cache
        with:
          path: |
            ~/.android/avd/*
            ~/.android/adb*
          key: avd-${{ runner.os }}-v2
      - name: Create AVD and generate snapshot for caching
        if: steps.avd-cache.outputs.cache-hit != 'true'
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 26
          force-avd-creation: false
          arch: x86_64
          profile: Nexus 6
          target: google_apis
          emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
          disable-animations: false
          script: echo "Generated AVD snapshot for caching."
      - name: Run tests
        uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 26
          force-avd-creation: false
          arch: x86_64
          profile: Nexus 6
          target: google_apis
          emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
          disable-animations: true
          script: python test/test.py

Maybe this is a problem very specific to my game engine or game, but wanted to ask if anybody saw something similar.

[murzagalin]

were you able to solve the problem?

[zommerfelds]

I started ignoring ocasional failures since the test is rather flaky. I'm not 100% sure if the problem is still around. I'll post here if I get another failed test.

[zommerfelds]

I had the problem again. Instead of fixing the root cause which I couldn't find I made my screenshot comparison less strict:

def image_compare(image_1, image_2):
    arr1 = np.array(image_1, np.int16)
    arr2 = np.array(image_2, np.int16)
    if arr1.shape != arr2.shape:
        raise "Image sizes don't match"

    diff_abs_max = np.max(np.abs(arr1 - arr2), axis=(0, 1))
    diff_stddev = np.std(arr1 - arr2, axis=(0, 1))

    # NOTE: In the CI emulator there is a lot of variation in the screenshot color.
    # I don't know why and I wasn't able to fix it. As a hack, we allow the images
    # to be slightly different:
    # Each of RGB values can't be more than 30 away from the golden, and the standard
    # deviation of the diff is less than 2 (out of 256).
    return max(diff_abs_max) < 30 and max(diff_stddev) < 2

[mrk-han]

Going to close this out because it's stale and seems like more of an issue with the emulator or the runner-image than something related to this action.