@TestPropertySource not recognizing yaml files

[DataWorm]

Spring Boot supports not only application.properties but also application.yml files for long time now and we are using it in all our projects for better readability. But when it comes to testing with annotations like @TestPropertySource then yaml files are not supported for whatever reason. If you try to specify a yml file as source, it does not complain but the test also won't be able to resolve the properties unless you are using the properties notation format and just store it with the .yml file extension.

I wonder why this is not supported yet. Is there another annotation I haven't seen yet or is there any plan to let TestPropertySource automatically recognize the file extension and make use of the corresponding yaml file loader?

[philwebb]

It's currently not supported because @TestPropertySource is a Spring Framework annotation and isn't aware of YAML. We've had a request to support it previously (#11921) which resulted in Framework issue that was ultimately closed. I wonder if we should try again for Framework 6.1.

@sbrannen any thought on this? Perhaps we could have a hook point to allow @TestPropertySource to deal with different file types?

[sbrannen]

@sbrannen any thought on this? Perhaps we could have a hook point to allow @TestPropertySource to deal with different file types?

Yes, I'm thinking about introducing a new Class<? extends PropertySourceFactory> factory() default PropertySourceFactory.class; attribute in @TestPropertySource analogous to the one already present in @PropertySource since Spring Framework 4.3. Though, I'll need to investigate the feasibility of such a change.

@philwebb and @wilkinsona, what do you think about that?

[sbrannen]

Though, I'll need to investigate the feasibility of such a change.

At a first glance, the biggest challenge I foresee with supporting a custom factory per @TestPropertySource annotation is that MergedContextConfiguration.getPropertySourceLocations() contains all merged locations from all such repeatable annotations, meaning there would not be any easy way to determine which factory to use for which files.

One option would be to use the first (or last) custom configured factory to process all locations -- which would require that all locations configured via @TestPropertySource on a given test class be the same type of file format.

Another option would be to create some sort of map from file extension to factory -- though I'm not sure how we would want to support that mapping via annotation-based configuration.

Perhaps instead of an external mapping, we could introduce the following default method in the PropertySourceFactory SPI and allow factory implementations to check the supported file extensions themselves (and fall back to using DefaultPropertySourceFactory).

default boolean supportsResource(@Nullable String name, EncodedResource resource) {
	return true;
}

Thoughts?

[wilkinsona]

I like the idea of aligning with @PropertySource but hadn't considered the complications introduced by MergedContextConfiguration.

I wonder if it would be possible to introduce a new method on MergedContextConfiguration that returns something richer than a String[]. That richer type would provide both the locations and the factory from @TestPropertySource. That feels a little cleaner to me than changing PropertySourceFactory in spring-core to accommodate a requirement that appears to be specific to spring-test.

[sbrannen]

I wonder if it would be possible to introduce a new method on MergedContextConfiguration that returns something richer than a String[].

I was also pondering introducing a new method in MCC.

That richer type would provide both the locations and the factory from @TestPropertySource.

Well, it turns out that we already have such a record as of Framework 6.0.

• org.springframework.core.io.support.PropertySourceDescriptor

So we could return a list of descriptors.

That feels a little cleaner to me than changing PropertySourceFactory in spring-core to accommodate a requirement that appears to be specific to spring-test.

Indeed.

[sbrannen]

• see https://github.com/spring-projects/spring-framework/issues/30981

[wilkinsona]

Thanks, @sbrannen.

Closing in favour of the Framework issue.

[sbrannen]

FYI: the feature is already available in Spring Framework 6.1 snapshots in case anyone wants to try it out.

https://github.com/spring-projects/spring-framework/commit/04cce0bafd21c3f6b8f427ab2eac75ffadefb585

[philwebb]

Reopening to see if we can provide a YAML PropertySourceFactory or perhaps an adapter to our own PropertySourceLoader interface.

[lfvJonas]

Any news on this issue?

[michaldo]

Please update status of this issue. Crawling all connected issues i found it is implemented but not released https://github.com/spring-projects/spring-boot/pull/42603#issuecomment-2517766306

The reason we didn't make that an official feature of the Spring TestContext Framework is that we doubted that it would meet the needs of enough developers

Let me explain my need then. I have a couple of closely related beans, which I can quickly test within small context, isolated from rest of application as well as isolated from infrastructure (DB) provided by Spring Boot. That is a reason why @SpringBootTest is not considered. I can do it with "fast" annotation @SpringJUnitConfig If I hardcode numbers, I will be done. Unfortunately, I made numbers configurable with defaults in application.yaml. Numbers are referenced by @Value("${...}) In test I have to copy each number to @TestPropertySource. It is tedious and stupid, because reasonable defaults are already present in application.yaml.

Therefore I think it is worth to release the feature. However, if the feature is is not planned to be released, I would appreciate a hint how to setup limited context of few beans and pass them application.yaml

PS. After reading comment to connected issues I understood that support for application.yaml is possible but in basic scope. Perfect for me, because my need is also very basic - do not copy-paste application.yaml to @TestPropertySource

[wilkinsona]

Why do you believe that it has been implemented but not released? The PR to which you linked (#42603) was declined so, from this repository's perspective, it has not yet been implemented. For that reason, this issue remains open.

[michaldo]

https://github.com/spring-projects/spring-framework/blob/main/spring-test/src/test/java/org/springframework/test/context/env/YamlTestPropertySourceTests.java

It works and is part of main branch, but crucial classes are present in test, not main

[wilkinsona]

Thanks. Those are tests for plugging in a custom factory when using TestPropertySource. This issue is tracking providing a factory for YAML in Spring Boot. Until it has been implemented in a way that we feel provides enough value to be generally useful and doesn't have too many surprising limitations, you can provide your own inspired by the implementation in Spring Framework's tests or in https://github.com/spring-projects/spring-boot/pull/42603#issuecomment-2625813248 if you're comfortable with the limitations.

[michaldo]

As a discussion is spread over many issues, it is not easy why implemented-not-released solution is not good enough.

Let's start with multi document support. It is told that it is not supported and it is expected to be supported. I think it is over-expectation, because multi document support does not work for *.properties now, why it should work for *.yaml?

Demo: ser/test/resources/multi-document.properties

# https://docs.spring.io/spring-boot/reference/features/external-config.html#features.external-config.files.multi-document
# For example, the following specifies that the second document is only active when running on Kubernetes, and only when either the “prod” or “staging” profiles are active:

myprop=always-set
#---
spring.config.activate.on-cloud-platform=kubernetes
spring.config.activate.on-profile=prod | staging
myotherprop=sometimes-set

src/test/java/demo/MultiDocumentTest.java

@TestPropertySource(locations = "/multi-document.properties")
class MultiDocumentTest {

    @SpringJUnitConfig
    @Nested class Vanilla {

        @Test
        void vanilla_was_never_working(@Autowired Environment env) {
            System.out.println("Vanilla $myotherprop: " + env.getProperty("myotherprop"));

        }
    }

    @SpringBootTest
    @Nested class Boot {

        @Test
        void boot_was_never_working(@Autowired Environment env) {
            System.out.println("Boot $myotherprop: " + env.getProperty("myotherprop"));
        }
    }
}

Vanilla $myotherprop: sometimes-set Boot $myotherprop: sometimes-set

I think that everything that is blamed on YamlPropertySourceFactory applies equally to DefaultPropertySourceFactory.