Introduce `@FieldSource` for parameterized tests

[ktkopone]

Overview

Introduce @FieldSource argument source for parameterized tests.

The feature is analogous to the existing @MethodSource support, except that the value of a named field is used as the source of arguments instead of the value returned by a factory method.

See https://github.com/junit-team/junit5/issues/2014#issuecomment-1523242161 and https://github.com/junit-team/junit5/issues/2014#issuecomment-1523333359 for examples.

Deliverables

• [x] Introduce @FieldSource and FieldArgumentsProvider.
  • [x] Reject fields with Stream values.
  • [x] Support Supplier<T> fields where T is a type supported for @MethodSource.
  • [ ] Write unit and integration tests for preconditions, edge cases, etc.
  • [ ] Provide examples, etc. in class-level Javadoc for @FieldSource.
• [ ] Document in the User Guide.
• [ ] Document in the Release Notes.

---

Original Issue Description

I understand that for memory/performance you probably want to recommend everyone use a Stream-returning method and @MethodSource, but given that an @ArraySource would be easier to use without having to convert all test data into methods, this would be a nice-to-have.

[marcphilipp]

Have you tried @ValueSource?

[sbrannen]

A @MethodSource factory method doesn't have to return a Stream. It can also return an array. So why don't you just do that?

Also, how would an @ArraySource annotation be used?

Would it reference the name of a field that has an array type?

If not, then @ValueSource likely already meets your needs as @marcphilipp already pointed out.

[ktkopone]

Apologies, I had a longer writeup with context but tried to pair it down. In my case I have an array variable of strings that I want to reuse as data for multiple parameterized tests (hence I don't want to just hardcode it in the @ValueSource strings parameter). If there's a way to pass an array variable to @ValueSource without running into Attribute value must be constant, I don't know it.

It's good to know that @MethodSource can return arrays. However, it is still a pain point that I'll have to write a custom method just to return data which I have stored in array vars. I could skip on having the array variable at all but I like being able to define all my test data together as properties of a TestData class, and I'm sure there exist scenarios where there would be good reasons to store in arrays rather than only being returned via methods.

So given that in many use cases the data will live in an array field, it would be convenient if the user could pass this via @ArraySource(arrayVar) without having to write a dummy method for each field.

[sbrannen]

Based on your feedback, what you are effectively requesting is a @FieldSource annotation analogous to the existing @MethodSource. In other words, there is nothing specific in your request that pertains only to arrays. I will therefore update the title of the issue accordingly.

[sbrannen]

Tentatively slated for the 5.6 backlog for team discussion

[marcphilipp]

In the meantime, this might be a little heavy on the ceremony but works:

@Retention(RetentionPolicy.RUNTIME)
@ValueSource(strings = { "racecar", "radar", "able was I ere I saw elba" })
@interface MyArraySource {}

@ParameterizedTest
@MyArraySource
void palindromes(String candidate) {
	assertTrue(StringUtils.isPalindrome(candidate));
}

[kriegfrj]

I just hit this issue - @marcphilipp's workaround is only useful if you don't intend on re-using the array of values in your code. If you want to access the array in your code you either have to duplicate the initialisation list somewhere in an array initialiser, or else you have to use reflection to get access to the annotation on your meta-annotated annotation. A @FieldSource would be nice; otherwise a @MethodSource can probably be used as a workaround.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.

[stale[bot]]

This issue has been automatically closed due to inactivity. If you have a good use case for this feature, please feel free to reopen the issue.

[ahertel]

private static final String[] TREATMENTS = {"1", "2"};
@ParameterizedTest
@ValueSource(strings = TREATMENTS)
public void test(String treatment) {}

This gives a "Attribute value must be constant" error.

How can i re-open this issue?

[sbrannen]

@ahertel, if you want to use the contents of a static array as the source for a parameterized test, you will currently have to use a factory method as in the following example.

private static final String[] TREATMENTS = {"1", "2"};

@ParameterizedTest
@MethodSource("treatments")
void test(String treatment) {}

static String[] treatments() {
    return TREATMENTS;
}

[ahertel]

@sbrannen thanks Sam. I discovered that this also works :)

private static final String T1 = "T1";
private static final String T2 = "T2";

@ParameterizedTest
@ValueSource(strings = {T1, T2})
public void test(String treatment) {}

After learning more about JUnit 5's offerings, I ended up just making an Enum and using @EnumSource

[sbrannen]

Lately I've noticed that there are times when it would be nice to be able to avoid the ceremony of creating a factory method for use with @MethodSource, and that brought me back to the idea of @FieldSource.

As a result, I have spiked an implementation which can be viewed here: https://github.com/sbrannen/junit5/commit/e74592a937eeba0415611831c740458628eb63de

In light of that, I am reopening this issue.

[sbrannen]

For those who missed the link in my previous comment, the following demo tests work.

class FieldSourceDemo {

	private static final String[] arrayOfStrings = { "apple", "banana" };

	private static final Object listOfStrings = List.of("apple", "banana");

	private static final List<String> reversedStrings = List.of("banana", "apple");

	private static final Stream<Arguments> namedArguments = Stream.of(//
		arguments(named("Apfel", "apple")), //
		arguments(named("Banane", "banana"))//
	);

	@ParameterizedTest
	@FieldSource // default field name
	void arrayOfStrings(String fruit) {
		assertFruit(fruit);
	}

	@ParameterizedTest
	@FieldSource("listOfStrings")
	void singleFieldSource(String fruit) {
		assertFruit(fruit);
	}

	@ParameterizedTest
	@FieldSource({ "listOfStrings", "reversedStrings" })
	void multipleFieldSources(String fruit) {
		assertFruit(fruit);
	}

	@ParameterizedTest
	@FieldSource
	void namedArguments(String fruit) {
		assertFruit(fruit);
	}

	@Nested
	@TestInstance(Lifecycle.PER_CLASS)
	class NestedTests {

		// Non-static field
		final List<String> strings = List.of("apple", "banana");

		@ParameterizedTest
		@FieldSource("strings")
		void nonStaticFieldSource(String fruit) {
			assertFruit(fruit);
		}

	}

	static void assertFruit(String fruit) {
		assertTrue("apple".equals(fruit) || "banana".equals(fruit));
	}

}

[sbrannen]

Update: the latest commit in the PoC adds support for referencing static fields in external classes via a fully-qualified field name, as demonstrated below.

@ParameterizedTest
@FieldSource("example.Utils#fruits")
void externalField(String fruit) {
	assertFruit(fruit);
}

[marcphilipp]

Team decision: Investigate more closely for 5.11. Reject fields with Stream values; support Supplier<T> fields where T is a type supported for @MethodSource, though.

[sbrannen]

Update

The latest work in my feature branch is effectively feature complete except documentation in the user guide and extensive tests.

[sbrannen]

Team decision: Investigate more closely for 5.11. Reject fields with Stream values; support Supplier<T> fields where T is a type supported for @MethodSource, though.

While writing exhaustive tests for this feature, I determined that we should also reject fields with Iterator values, since an Iterator would also be consumed after the first retrieval of the field and therefore invalid for potential subsequent retrievals.

[sbrannen]

Update

The latest work in my feature branch is now feature complete and fully tested.

All that remains is documentation in the user guide and release notes.

[lukeu]

Huh nice timing to search on a feature and find it just implemented. :-)

I'd just like to say that what I really love about this is that it will make data look like data, and leave methods to implement behaviour. IMHO this much more than just a minor cosmentic difference, I think it will really accelerate "visually parsing" and understanding tests.