bazel icon indicating copy to clipboard operation
bazel copied to clipboard

Set default test size to "small"

Open matts1 opened this issue 1 year ago • 0 comments

Description of the feature request:

If unspecified, a test's size currently defaults to "medium". I think small is a far more reasonable default.

aspect_bazel_lib says:

The [test-encyclopedia](https://bazel.build/reference/test-encyclopedia) says:
* Tests may return arbitrarily fast regardless of timeout.
* A test is not penalized for an overgenerous timeout, although a warning may be issued:
* You should generally set your timeout as tight as you can without incurring any flakiness.

However Bazel's default for timeout is medium, which is dumb given this guidance.

I'm inclined to agree with this. The vast majority of tests should be marked as small, so it would be more convenient, and given the guidance, it would make sense to set the timeout as tight as is allowed (small) by default, and then bazel will timeout and tell you to increase the timeout.

To achieve this, I request creating a flag --incompatible_default_test_size and default it to medium. Then in a future release, we can change it to small, before eventually deleting the flag.

Which category does this issue belong to?

Java Rules

What underlying problem are you trying to solve with this feature?

Our pre-upload checks add the flag --test_size_filters=small. I just recently ran into an issue where the size of a test was not specified, and so the test was skipped. I think defaulting it to small is far more useful, and also means that users don't have to remember to set the test size - instead, they will know to set the test size because the test fails from timing out.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

7.1.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

matts1 avatar May 13 '24 22:05 matts1