Sometimes the ErrorBoundary test fails locally for me with strange line numbers

[mstange]

Summary of all failing tests FAIL src/test/components/ErrorBoundary.test.js ● app/ErrorBoundary › matches the snapshot

    expect(received).toMatchSnapshot()

    Snapshot name: `app/ErrorBoundary matches the snapshot 1`

    - Snapshot  - 1
    + Received  + 1

    @@ -27,11 +27,11 @@
            <div>
              Error: This is an error.
            </div>
            <div>
              
    -     at ThrowingComponent (REDACTED)/src/test/components/ErrorBoundary.test.js:20:11)
    +     at ThrowingComponent (REDACTED)/src/test/components/ErrorBoundary.test.js:30:11)
          at ErrorBoundary (REDACTED)/src/components/app/ErrorBoundary.js:28:66)
          at LocalizationProvider (REDACTED)/node_modules/@fluent/react/index.js:(REDACTED)
            </div>
          </div>
        </div>

      54 |   });
      55 |
    > 56 |   it('shows the error message children when the component throws error', () => {
         |                                  ^
      57 |     const { getByText } = setupComponent(<ThrowingComponent />);
      58 |     expect(getByText(friendlyErrorMessage)).toBeInTheDocument();
      59 |   });

      at Object.<anonymous> (src/test/components/ErrorBoundary.test.js:56:34)

  ● app/ErrorBoundary › reports errors to the analytics

    expect(jest.fn()).toHaveBeenCalledWith(...expected)

    - Expected
    + Received

      "send",
      "exception",
      Object {
    -   "exDescription": StringMatching /Error: This is an error\.\n\n    at ThrowingComponent \(.*[/\\]ErrorBoundary.test.js:20:11\)\n    at ErrorBoundary \(.*[/\\]ErrorBoundary.js:28:66\)\n    at LocalizationProvider \(.*[/\\]@fluent[/\\]react[/\\]index.js:.*\)/,
    +   "exDescription": "Error: This is an error.
    +
    +     at ThrowingComponent (/Users/mstange/code/perf.html/src/test/components/ErrorBoundary.test.js:30:11)
    +     at ErrorBoundary (/Users/mstange/code/perf.html/src/components/app/ErrorBoundary.js:28:66)
    +     at LocalizationProvider (/Users/mstange/code/perf.html/node_modules/@fluent/react/index.js:111:26)",
        "exFatal": true,
      },

    Number of calls: 1

      87 |           new RegExp(stripIndent`
      88 |               Error: This is an error\\.
    > 89 |
         | ^
      90 |                   at ThrowingComponent \\(.*[/\\\\]ErrorBoundary.test.js:20:11\\)
      91 |                   at ErrorBoundary \\(.*[/\\\\]ErrorBoundary.js:28:66\\)
      92 |                   at LocalizationProvider \\(.*[/\\\\]@fluent[/\\\\]react[/\\\\]index.js:.*\\)

      at callback (src/test/components/ErrorBoundary.test.js:89:23)
      at withAnalyticsMock (src/test/fixtures/mocks/analytics.js:13:3)
      at Object.<anonymous> (src/test/components/ErrorBoundary.test.js:87:38)

 FAIL  src/test/components/MenuButtons.test.js
  ● app/MenuButtons › <Publish> › matches the snapshot for the opened panel for a nightly profile

    expect(received).toMatchSnapshot()

    Snapshot name: `app/MenuButtons <Publish> matches the snapshot for the opened panel for a nightly profile 1`

    - Snapshot  - 1
    + Received  + 1

    @@ -84,11 +84,11 @@
          <div
            class="menuButtonsPublishButtons"
          >
            <a
              class="photon-button menuButtonsPublishButton menuButtonsPublishButtonsDownload"
    -         download="Firefox 1970-01-01 00.00 profile.json.gz"
    +         download="Firefox 1969-12-31 19.00 profile.json.gz"
              href="mockCreateObjectUrl"
            >
              <span
                class="menuButtonsPublishButtonsSvg menuButtonsPublishButtonsSvgDownload"
              />

      255 |       const { profile } = createSimpleProfile('nightly');
      256 |       const { getPanel, openPublishPanel } = setupForPublish(profile);
    > 257 |       await openPublishPanel();
          |                          ^
      258 |       expect(getPanel()).toMatchSnapshot();
      259 |     });
      260 |

      at Object.<anonymous> (src/test/components/MenuButtons.test.js:257:26)

  ● app/MenuButtons › <Publish> › matches the snapshot for the opened panel for a release profile

    expect(received).toMatchSnapshot()

    Snapshot name: `app/MenuButtons <Publish> matches the snapshot for the opened panel for a release profile 1`

    - Snapshot  - 1
    + Received  + 1

    @@ -79,11 +79,11 @@
          <div
            class="menuButtonsPublishButtons"
          >
            <a
              class="photon-button menuButtonsPublishButton menuButtonsPublishButtonsDownload"
    -         download="Firefox 1970-01-01 00.00 profile.json.gz"
    +         download="Firefox 1969-12-31 19.00 profile.json.gz"
              href="mockCreateObjectUrl"
            >
              <span
                class="menuButtonsPublishButtonsSvg menuButtonsPublishButtonsSvgDownload"
              />

      266 |     });
      267 |
    > 268 |     it('matches the snapshot for the menu buttons and the opened panel for an already uploaded profile', async () => {
          |                          ^
      269 |       const { profile } = createSimpleProfile();
      270 |       const { getPanel, container, navigateToHash, openPublishPanel } =
      271 |         setupForPublish(profile);

      at Object.<anonymous> (src/test/components/MenuButtons.test.js:268:26)

  ● app/MenuButtons › <Publish> › matches the snapshot for the menu buttons and the opened panel for an already uploaded profile

    expect(received).toMatchSnapshot()

    Snapshot name: `app/MenuButtons <Publish> matches the snapshot for the menu buttons and the opened panel for an already uploaded profile 2`

    - Snapshot  - 1
    + Received  + 1

    @@ -79,11 +79,11 @@
          <div
            class="menuButtonsPublishButtons"
          >
            <a
              class="photon-button menuButtonsPublishButton menuButtonsPublishButtonsDownload"
    -         download="Firefox 1970-01-01 00.00 profile.json.gz"
    +         download="Firefox 1969-12-31 19.00 profile.json.gz"
              href="mockCreateObjectUrl"
            >
              <span
                class="menuButtonsPublishButtonsSvg menuButtonsPublishButtonsSvgDownload"
              />

      281 |         setupForPublish(profile);
      282 |       await openPublishPanel();
    > 283 |       expect(queryPreferenceCheckbox()).toBeTruthy();
          |                          ^
      284 |     });
      285 |
      286 |     it('does not show the Include preference values checkbox when a PreferenceRead marker is in the profile', async () => {

      at Object.<anonymous> (src/test/components/MenuButtons.test.js:283:26)


Snapshot Summary
 › 4 snapshots failed from 2 test suites. Inspect your code changes or run `yarn run test-debug -u` to update them.

Test Suites: 2 failed, 106 passed, 108 total
Tests:       5 failed, 2 skipped, 4 todo, 1493 passed, 1504 total
Snapshots:   4 failed, 295 passed, 299 total
Time:        53.898 s, estimated 83 s

┆Issue is synchronized with this Jira Task

[mstange]

Today these tests are passing for me again. I had to run yarn install due to the core-js dependency update, maybe this invalidated some cache. Though I did try yarn install yesterday (after deleting the node_modules directory) and it didn't help.

[julienw]

The datetime errors on the download attribute sounds like a problem with locales. It looks like this is UTC-5 (is that toronto?) I get the same errors with LC_ALL=C TZ="America/Toronto" NODE_ENV=test ./node_modules/.bin/jest menubuttons.test.

Rough idea: you ran tests with yarn jest (or some tool runs jest directly) instead of yarn test?

I'm still not sure about the problem in ErrorBoundary's line. I would think that maybe different babel configuration could maybe transform the source code differently, and give different lines because of that. But I'm not sure really how that could happen when running tests with yarn test.