Verifier.verify_with_broker reports success even if the provider_states_setup_url returns a 500 as part of its setup and fails to run a particular case

[joshua-badger]

I expect that if a provider state fails to setup and that URL returns a 500, I would expect the verifier test to have failed/erred due to being unable to setup a state correctly. As shown in the test results above, no such thing happened. Are my expectations correct here?

The test is as follows (run in pytest):

def test_pact_verification(
    pact_verifier,
    live_server,
    authentication_setup,
    external_services_mocks,
    contract_test_settings,
    provider_states_setup,
    settings,
):
    # set up downstream pacts that get used in this test?

    success, logs = pact_verifier.verify_with_broker(
        broker_url=<redacted>,
        publish_version=<redacted>,
        publish_verification_results=True
        verbose=True,
        provider_states_setup_url=provider_states_setup,
    )
    assert success == 0