Scala Common Enrich: mock history.openweathermap.org in the test to avoid transient errors

[knservis]

The test com.snowplowanalytics.snowplow.enrich.common.enrichments.registry.WeatherEnrichmentSpec

depends on an external service (history.openweathermap.org). This can fail for various reasons. We should ensure that the test doesn't fail if the service is unavailable (or the query quota is exceeded). The way to do that is more than likely to mock that service.

[alexanderdean]

Seems wise!

[chuwy]

There's mocks inside scala-weather. API access layer under this enrichment.

OWM as many other 3rd-party can change format of output data and this spec tests exactly that - we want to be sure that we're on the same page with OWM.

However, I agree that it's not a best approach and weather enrichment is not a special citizen - 3rd party data providers can break adapters as well. I can imagine a separate module with scheduled tests testing all 3rd-party data provider's output in one place.

So, there's definitely something we can explore regarding that, but mocks cannot solve the problem, unfortunately.

[knservis]

In which case the test should not fail when there is a connection timeout. To me, when I am testing something unrelated and the whole suite fails because of an external service timeout, something is wrong. Is the OWM API not versioned?

[chuwy]

In which case the test should not fail when there is a connection timeout.

If their service suddenly becomes retired - we will want to know about it as well. This is another goal of this test.

Is the OWM API not versioned

No, it is not. They claim they're, but also they prove the opposite all the time - I have not much trust for them.