test-runner [Feature] Allow preventing smoke-tests on particular stories

Describe the Feature

I have a Themed story in many of my stories files which simply renders other stories together in a "light mode" and "dark mode". If I'm testing each of those other stories, then I would like to save the time and prevent my Themed stories from being smoke-tested. As far as I know, there's no way to do this currently.

One approach might be to add a parameter, like

const Themed = {
  ...myCSF,
  parameters: {
    testing: {
      disable: true,
    },
  },
}

If this approach was taken, it might also be a way to set the it description of the test (see https://github.com/storybookjs/test-runner/issues/71), even though it's a little clunky. In addition, maybe it would be a cool way to add setup/teardown hooks for a particular story. Even though I haven't found that I need those so far, but I think that's because each test is a fresh page load. Which is great for isolation, but definitely slows things down. And the individual stories themselves are not new page loads, so there could need to be setup/teardown between them. But that's probably a separate issue.

Mar 07 '22 20:03 IanVS

I have another use case for needing to prevent test-runner from executing a particular story. This time, it's because of my play function. It executes a click on an anchor tag that causes navigation (only a hash change, so it doesn't actually break the page). Puppeteer still detects this as a navigation, and dies with an error of Execution context was destroyed, most likely because of a navigation.. If I could disable the story from the test runner, I could still snapshot it with chromatic, which would meet my needs.

Mar 23 '22 15:03 IanVS

Here's another situation where I need to disable a particular test. I have a story which sets a narrow viewport, which exposes a mobile menu button, which I use the play function to click on, and then I snapshot the result with chromatic.

However, this doesn't work in the test runner, because there's no way to set the viewport size. (https://github.com/storybookjs/test-runner/issues/85). Since I also can't disable the test-runner, I think I'll just need to weaken my play function to be sure that it doesn't error when there's no mobile menu button (use queryBy instead of findBy, for instance). But it would be better if I could just disable the test (until #85 is solved).

Apr 06 '22 16:04 IanVS

I'd also love to be able to easily opt-out stories from test-runner as well, and I like the use of a parameter similar to how Storyshots allows this via storyshots: { disable: true }.

I did find a way to achieve just that via a custom decorator.

Add this decorator to your preview.* file:

DisableTestRunnerDecorator: DecoratorFn = (Story, { parameters }) => {
  if (parameters.testRunner?.disable === true && navigator.userAgent === 'Storybook Test Runner') {
    return <>Disabled for Test Runner</>;
  }

  return <Story />;
};

export const decorators: DecoratorFn[] = [
  DisableTestRunnerDecorator,
  // any other decorators you might have
];

Then (use test-storybook --eject and) add the following to your test-runner-jest-config.js:

const { getJestConfig } = require('@storybook/test-runner');

// The default configuration comes from @storybook/test-runner
const defaultConfig = getJestConfig()

module.exports = {
  ...defaultConfig,
  /** Add your own overrides below
   * @see https://jestjs.io/docs/configuration
   */
  testEnvironmentOptions: {
    'jest-playwright': {
      ...defaultConfig['jest-playwright'],
      contextOptions: {
        userAgent: 'Storybook Test Runner',
      }
    }
  }
};

Jun 24 '22 09:06 FokkeZB

What about skipping the render if the preRender hook returned false or something like that? That would allow people to inspect context / parameters and skip based on that?

My use case is to exclude stories that have { chromatic: { disableSnapshot: true } } set, from the coverage report.

Nov 13 '22 12:11 nathanpower

@FokkeZB in Storybook v7 it seems the condition has changed for the userAgent. It's required to do:

  if (parameters.testRunner?.disable === true && navigator.userAgent.includes('StorybookTestRunner')) {
    return <>Disabled for Test Runner</>;
  }

_(the user agent sent is now Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/109.0.5414.46 Safari/537.36 ([email protected]))

@yannbf any plan for this officially?