Pact jvm issue with consumer filtering

[rarora-trip]

I am facing issue related to consumer filtering. I have multiple consumer(say C1 and C2) for one provider. Both consumer publish their individual contracts. When feature branch of C1 publishes new contract, webhook triggers build for provider with pact Url and consumer name environment variables for AllowOverridePactUrl annotation. When provider build runs it fails with Provider state call back failed for C2.

Below error log shows filter consumer name is “C1” and has pact url but its still trying to find state in “C2” provider test. It should check in C1ToP1ContractTest C2ToP1ContractTest > C1 - GET Product by Product Code STANDARD_OUT [06:17:38,247][INFO] provider.junit5.PactVerificationStateChangeExtension - Invoking state change method 'GetByProductCode':SETUP Verifying a pact between C1 and P1 [from Pact Broker <BROKER>/pacts/provider/product-service/consumer/C1/pact-version/9ba7240b86a1ad9f4900fa75e3dab5619744a4e5/metadata/Y3ZuPTk1MTJlNzIzNGEzMjdhYjg4ZjZkMjMxNTA5ZWE2NWUwYjliZTRkMGQmY3Z0W109ZmVhdHVyZV9TVVBQTFktODMxNS1za2lwLW5vbi1zYXBpLXByb2R1Y3RzLXdoZW4tZW5hYmxlLXNjaGVkdWxpbmctcGFjJnc9dHJ1ZQ] Given GetByProductCode GET Product by Product Code [06:17:38,270][WARN] api.text.TextContentHelper - ContentRef not found :: {"contentRef":"CR-REF1"} returns a response which has status code 200 (OK) has a matching body (OK) [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - Not all of the 7 were verified. The following were missing: [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - Get First Publish Requested User By product code [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - GET Supplier by Product Code [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - Lookup Product Details [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - GET product code request [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - GET products status [06:17:38,277][WARN] pact.provider.DefaultTestResultAccumulator - GET Supplier by Product Code C2ToP1ContractTest > C1 - Get First Publish Requested User By product code STANDARD_OUT [06:17:38,279][ERROR] provider.junit5.PactVerificationStateChangeExtension - Provider state change callback failed au.com.dius.pact.provider.junitsupport.MissingStateChangeMethod: Did not find a test class method annotated with @State("GetFirstPublishRequestedUser") for Interaction "Get First Publish Requested User By product code" with Consumer "C1" at au.com.dius.pact.provider.junit5.PactVerificationStateChangeExtension.invokeStateChangeMethods(PactVerificationStateChangeExtension.kt:125) ~[junit5-4.1.32.jar:4.1.32] Please suggest how to fix it

version: 'au.com.dius.pact.provider:junit5:4.1.32' (also tried 4.2.8 and faced the same issue)

Provider class annotations @Provider("P1") @VerificationReports @PactBroker(consumerVersionSelectors = {@VersionSelector(consumer = "C1", tag = "develop")}) @AllowOverridePactUrl public class C1ToP1CT {

please find additional details at: https://pact-foundation.slack.com/archives/C9UN99H24/p1658221653637689

[rholshausen]

Can you try adding the @Consumer annotation to the different tests?

[rarora-trip]

I did, still same issue

[rholshausen]

I was able to replicate this issue, and then resolve it by adding the @Consumer and @IgnoreNoPactsToVerify annotations to the different tests.

I.e.,

For consumer 1:

@RunWith(SpringRestPactRunner.class)
@Provider("spring-boot-maven-provider")
@Consumer("consumer1") // <--- Required
@PactBroker(url = "https://testdemo.pactflow.io/",
        authentication = @PactBrokerAuth(token = "XXXXXXXXXXXXXXXXX"),
        consumerVersionSelectors = {@VersionSelector(consumer = "consumer1", tag = "develop")}
)
@AllowOverridePactUrl
@IgnoreNoPactsToVerify // <--- Required
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public class C1PactTest

and the for consumer 2:

@RunWith(SpringRestPactRunner.class)
@Provider("spring-boot-maven-provider")
@Consumer("consumer2") // <--- Required
@PactBroker(url = "https://testdemo.pactflow.io/",
        authentication = @PactBrokerAuth(token = "XXXXXXXXXXXXXXXXX"),
        consumerVersionSelectors = {@VersionSelector(consumer = "consumer2", tag = "develop")}
)
@AllowOverridePactUrl
@IgnoreNoPactsToVerify // <--- Required
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public class C2PactTest

[rarora-trip]

This seems like a workaround. We should not require IgnoreNoPactsToVerify annotation. Also, consumer details are already present in VersionSelectors. I removed Consumer annotation when i upgraded to VersionSelectors. It seems like AllowOverridePactUrl is not working with VersionSelectors.

I will give it a go.

[rholshausen]

The problem is that all the tests will run, and the tests need to know which consumer they are for, so they know not to try validate the pact, and also not to fail if the consumer is not for that test. The selectors only tell the test how to query the broker, not which consumer the test is for.

If you use a single test class, you will not have this issue.

[rarora-trip]

I tried above fix but got stuck into next issue. Now, there is one consumer and two providers, with separate pact between C and P1, C and P2. When Consumer C publishes pact, it triggers provider build with separate pactUrl for both P1 and P2. In our case P1 and P2 are part of same build, so two build with different pact url are triggered.

Since consumer is same, when build is triggered for provider P1, provider P2 also tries to verifies test and fails with

CToP2ContractTest > pactVerificationTestTemplate(PactVerificationContext)[1] STANDARD_OUT [05:25:51,523][ERROR] provider.junit5.PactVerificationStateChangeExtension - Provider state change callback failed au.com.dius.pact.provider.junitsupport.MissingStateChangeMethod: Did not find a test class method annotated with @State("method related to P1")

Provider Test Class 1: @Provider("P1") @VerificationReports @PactBroker(consumerVersionSelectors = {@VersionSelector(consumer = "C", tag = "develop", fallbackTag = "develop-SNAPSHOT")}) @AllowOverridePactUrl @Consumer("C") @IgnoreNoPactsToVerify public class CToP1ContractTest

ProviderTestClass2: @Provider("P2") @VerificationReports @PactBroker(consumerVersionSelectors = {@VersionSelector(consumer = "C", tag = "develop", fallbackTag = "develop-SNAPSHOT")}) @AllowOverridePactUrl @Consumer("C") @IgnoreNoPactsToVerify @ExtendWith(ServiceTest.class) public class CToP2ContractTest

My request is, since pact url knows which consumer and provider are reason for trigger, Provider test class has annotation for both consumer and provider, there should not be confusion for which tests to run

[praveen-em]

I was able to replicate this issue, and then resolve it by adding the @Consumer and @IgnoreNoPactsToVerify annotations to the different tests.

I.e.,

For consumer 1:

@RunWith(SpringRestPactRunner.class)
@Provider("spring-boot-maven-provider")
@Consumer("consumer1") // <--- Required
@PactBroker(url = "https://testdemo.pactflow.io/",
        authentication = @PactBrokerAuth(token = "XXXXXXXXXXXXXXXXX"),
        consumerVersionSelectors = {@VersionSelector(consumer = "consumer1", tag = "develop")}
)
@AllowOverridePactUrl
@IgnoreNoPactsToVerify // <--- Required
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public class C1PactTest

and the for consumer 2:

@RunWith(SpringRestPactRunner.class)
@Provider("spring-boot-maven-provider")
@Consumer("consumer2") // <--- Required
@PactBroker(url = "https://testdemo.pactflow.io/",
        authentication = @PactBrokerAuth(token = "XXXXXXXXXXXXXXXXX"),
        consumerVersionSelectors = {@VersionSelector(consumer = "consumer2", tag = "develop")}
)
@AllowOverridePactUrl
@IgnoreNoPactsToVerify // <--- Required
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
public class C2PactTest

We have similar filtering issue for webhook triggered runs. Looks like using @Consumer will resolve the issue. However, we have the same test (with certain states) that applies to more than one consumer. Is there a way to select multiple consumers using that annotation or any other alternate solution to avoid duplicating the tests for each consumer in this case? @rholshausen

[rholshausen]

No, when the webhook triggers, it will pass through the Pact for the event. If you have tests setup for consumer1 and consumer2, what will you expect to happen when someone publishes a Pact for consumer3?

By trying to setup different provider verification tests for different consumers, you are fighting against the framework.