Evaluate approaches for testing individual components headlessly

[mattsolo1]

Enzyme?

[phildarnowsky-broad]

@mattsolo1 @rileyhgrant I've been looking into this, and based largely on the recommendations in the React docs, I'd suggest that we stick to the simpler functionality of jest and jsdom when possible. We'll want to test more complex interactions as well, and where formerly we might have used Enzyme, the tool now favored for that is React Testing Library (where do they get these crazy names), see https://testing-library.com/docs/react-testing-library/intro. There is a lot of worthwhile stuff on that page and also https://reactjs.org/docs/testing.html.

Side note on end-to-end tests: contrary to my understanding of best practices, these docs also do recommend end-to-end tests against live hosts, as opposed to my preferred method of stubbing out e.g. the gnomAD API in our case. This made me think about whether my own position here was sound, and the conclusion I came to is that there's a time and place for both kinds of end-to-end test.

The chief advantage of stubbed end-to-end tests is that they're more reliable: they will throw fewer spurious test failures due to network glitches and other random factors that don't really have anything to do with the test. I've mentioned this before, but I really cannot overstate the pernicious effects of getting too comfortable with tests that sometimes pass and sometimes fail for no important reason.

The chief advantage of live end-to-end tests are that they are more realistic. No piece of software anywhere runs in isolation, and despite any efforts to the contrary, a prod environment in its full complexity can have significant differences from a dev or test environment, certainly enough to lead to nasty surprises at deploy time.

So my suggestion is this: for the purposes of our main test suite, the one that we run over and over again on our local machines, the one that gatekeeps PR approval, we stick to stubbed end-to-end tests. It would also be smart to develop some live ones, to come in somewhere later in the process, but here my thoughts are less fully formed. I'll mull that over some more over the weekend and perhaps we can all brainstorm a bit together next week, assuming that fits Matt's baby's plans.