pattern-directory icon indicating copy to clipboard operation
pattern-directory copied to clipboard

e2e test environment for patterns

Open tellyworth opened this issue 4 years ago • 2 comments

Following on from https://github.com/WordPress/pattern-directory/issues/22#issuecomment-773435840:

It seems like we need something a bit like a lightweight version of the block plugin e2e tests, that would run a few simple server-side JS tests to check a pattern for validity. And perhaps grab screenshots or other info.

How could we do this in a way that uses our own servers, or minimal additional server resources?

tellyworth avatar Feb 09 '21 21:02 tellyworth

The majority of the setup time in the existing e2e tests has been bootstraping the environment, actually running the JS to parse/validate on a shared pre-setup site should be possible with minimal extra resources needed.

Generating screenshots is more complicated however, as that requires more than just a JS runtime, but a browser runtime too. Now that I say that, I'm not even entirely sure if just a JS runtime is enough to run Gutenberg?

The largest hurdle that I can think of is the security aspect, we don't want to have any execution of user-supplied JS (Blocks the pattern uses, etc) outside of a "secure sandboxed" environment.

dd32 avatar Feb 12 '21 02:02 dd32

For the pattern validation task, we shouldn't need a faux-browser environment (or even a WordPress site), just a JS env for Jest should be enough. I was thinking something like the parser / validator tests — pass in the content string, check that it parses & doesn't return a missing block, etc.

I don't think we need the screenshots for patterns, since we'll be showing live previews of those.

The largest hurdle that I can think of is the security aspect, we don't want to have any execution of user-supplied JS (Blocks the pattern uses, etc) outside of a "secure sandboxed" environment.

The initial pattern directory won't load 3rd party blocks, only core blocks, so for now we can push that off -- but I think the Jest run, which is started and then terminated for each pattern, and doesn't run a browser env, should be enough of a "sandbox"

ryelle avatar Feb 12 '21 17:02 ryelle