feat: `deno test --include [glob/pattern]`

[ycmjason]

Hi,

I'd like to propose to add a --include [glob/pattern] CLI option for deno test.

Problem

My project currently have unit, integration and e2e tests. Unit tests live under src/**/*.test.ts, integration integration/**/*.test.ts and e2e e2e/**/*.test.ts.

Currently I rely on these tasks in deno.json to run them:

{
    "test:unit": "deno test --special-flags-for-unit src",
    "test:integration": "deno test --special-flags-for-integration integration",
    "test:e2e": "deno test --special-flags-for-e2e e2e"
}

Ideally, I'd like to just do

deno task test:integration path/to/integration.test.ts path/to/integration-2.test.ts

And have the arguments passed down to deno test. However, since the path integration is already specified in the test:integration task, all integration tests are run as a result.

Hence, to target specific test files during development, we ended up needing to keep copy-and-pasting deno test --some-special-flags.

I have also considered adding something like the following, omiting the file path.

"test:integration:prefix": "deno test --special-flags-for-integration integration",

This allow us to target files nicely, but forgetting to do so would mean running ALL test files, including unit and e2e using the special flags for integration.

Proposed solution

Add a --include [glob/pattern] option in deno test. I imagine this flag would basically change the default {*_,*.,}test.{js,mjs,ts,mts,jsx,tsx} glob for finding test files.

With --include, the problem mentioned above can be easily solved with tasks that look like this

{
    "test:unit": "deno test --special-flags-for-unit --include=src/**/*.test.ts",
    "test:integration": "deno test --special-flags-for-integration --include=integration/**/*.test.ts",
    "test:e2e": "deno test --special-flags-for-e2e e2e --include=e2e/**/*.test.ts"
}

Without specifying any paths, deno test should find test files based on the --include option from the CWD.

When the path is specified, for example deno task test:unit path/to/unit.test.ts, only the targetted path would run.

---

I haven't found useful option from https://docs.deno.com/runtime/reference/cli/test/ or any issues requesting for this feature yet, so I thought maybe I would propose it here.

Here's my project's deno.json for reference.

[bartlomieju]

I believe what you're asking for is already supported - deno test accepts a global pattern(s) as a free argument - deno test "integration/**/*.test.ts" will only run the files matching the glob. (Notice I put glob in quotes so that it's passed verbatim to Deno and not expanded by your shell)

[ycmjason]

@bartlomieju

Thanks for the prompt reply.

I have tried changing the task script to

    "test:unit": "deno test --some-special-flags \"src/**/*.test.ts\"",

Then run deno task test:unit path/to/test.ts. All unit tests are executed.

Am I misunderstanding what you have mentioned?

[bartlomieju]

Maybe I'm misunderstanding what you're trying to do. By default you want to have deno task test:unit execute all unit tests, but if you do deno task test:unit path/to/test.ts you only want to focus that single file and ignore the glob from task definition?

[ycmjason]

Maybe I didn't do a good job describing the issue.

Please allow me to clarify:

When putting the glob pattern in test:unit, deno task test:unit will run all matched paths. I think this is great! 👍

The issue we are having is that it wouldn't allow targeting specific file. Removing the glob pattern, however, would mean that all tests are run with the flags defined in the task if the target paths are not specified.

Ideally we'd love to see

• deno task test:unit to run all unit tests under src, defined by the --include pattern.
• deno task test:unit path/to/target/test.ts run only the specified path.

I hope this clarify things. 😊

Thanks a lot for the good work here! ❤️

[ycmjason]

Maybe I'm misunderstanding what you're trying to do. By default you want to have deno task test:unit execute all unit tests, but if you do deno task test:unit path/to/test.ts you only want to focus that single file and ignore the glob from task definition?

Yes! Basically! But I do think the current behaviour is very intuitive, i.e. passing a glob as an argument should run all files.

That's why I proposed the --include option.

[bartlomieju]

Unfortunately this wouldn't work - everything that is passed after "free args" is treated as args passed to Deno.args and usable in your program.

I can see usefulness of this solution, but I'm not yet sure how we could tackle that without breaking existing workflows.

I think your best bet for now is to create a custom script (or a task) that calls out to deno test subprocess an passes either the default glob or the provided glob.

Something along these lines:

{
    "tasks": {
        "test:unit": "deno run test_runner.ts"
    }
}

// test_runner.ts
let glob = Deno.args[0];

if (!glob) {
    glob = "src/**/*.test.ts";
}

new Deno.Command(Deno.execPath(), {
    args: ["test", "--some-special-flag", glob],
}).output();

[ycmjason]

@bartlomieju

I can see usefulness of this solution, but I'm not yet sure how we could tackle that without breaking existing workflows.

I understand there might be constraints I'm not seeing. Could you help me understand how adding an option to deno test might impact existing workflows? I'd love to better grasp the potential challenges we're facing here.

[bartlomieju]

I understand there might be constraints I'm not seeing. Could you help me understand how adding an option to deno test might impact existing workflows? I'd love to better grasp the potential challenges we're facing here.

Adding an option is not a problem - the problem is that the option must be specified before the "free arg", that is:

$ deno test --include="path/to/target/test.ts" ""src/**/*.test.ts"

would work - because --include is passed before the "free arg" with the default glob, but:

$ deno test ""src/**/*.test.ts" --include="path/to/target/test.ts"

will not work, because in this case --include="path/to/target/test.ts" will be passed as an argument to the script, available in Deno.args:

console.log(Deno.args);

[`--include="path/to/target/test.ts"`]

[ycmjason]

The solution I have in my mind is to use --include for the glob and free args for the target.

For example

# find all files based on src/**/*.test.ts glob
$ deno test --include="src/**/*.test.ts"

# find all files based on src/**/*.test.ts glob, but only under src/module-1
$ deno test --include="src/**/*.test.ts"  src/module-1

# find all files that matches both glob
$ deno test --include="src/**/*.test.ts" src/module-*

# only run that test, if the path matches --include
$ deno test --include="src/**/*.test.ts" path/to/target/test.ts

I think these are some similar options in vittest and jest:

• https://vitest.dev/config/#include
• https://jestjs.io/docs/cli#--testpathpatternregex

[ycmjason]

With the above, we could define the deno task as

"test": "deno test --include=\"src/**/*.test.ts\""

Then deno task test would run deno test --include=\"src/**/*.test.ts\", all files matching src/**/*.test.ts would be executed.

deno task test src/path/to/test.ts would then just run src/path/to/test.ts.

I hope this clarifies the intended usage of this option better!

Thanks @bartlomieju