enrich

enrich copied to clipboard

If an artifact for enrichments can't be downloaded (e.g. `"uri": "s3://snowplow-hosted-assets-us-east-1/third-party/ua-parser/"` for Stream Enrich in `eu-west-1`), Stream Enrich doesn't crash, doesn't process events, and keeps logging indefinitely : ``` WARN...

benjben

Common: improve derived_tstamp/implement better approximation of actual time of event

5

Currently, the derived timestamp logic is as follows: `derived_tstamp = collector_tstamp - (dvce_sent_tstamp - dvce_created_tstamp)` As I understand it, the logic is that the collector timestamp is the first time...

chuwy

**Project**: Stream Enrich **Version**: 1.1.0 **Expected behavior**: Credentials provider is not compatible with KCL v2. **Actual behavior**: Use the latest library. **Steps to reproduce**: I am on my way to...

szareiangm

Common: run JavaScript Enrichment after all other derived contexts have been added to the event

1

That way the user's JS can access derived contexts such as the ua-parser context. What do you think @alexanderdean ?

chuwy

Common: make it possible to filter fields outputted by the ip lookups enrichment

1

chuwy

Stream: duplicated conditions in getProvider methods from KinesisSink.scala and KinesisEnrich.scala

1

**Projects**: * Collectors * Enrich **Version**: Master **Expected behavior**: Log warning message in case of `accessKey` **or** `secretKey` not being set **Actual behavior**: Never logging anything regarding `accessKey` **or** `secretKey`...

chuwy

Common: explore user-defined enrichments

1

First attempt: https://github.com/snowplow/snowplow/commit/61ad6f9bfaea7299b21d579289782e54f59c141d#commitcomment-13187113 The current idea is to have users create a class extending `IUserEnrichment` defining a method which takes an `EnrichedEvent` and returns `Unit`, i.e. it mutates the `EnrichedEvent`...

chuwy

Common: add functionality to load files from DynamoDB or the local filesystem

1

chuwy

Common: add ability to match internal domains based on a regex

1

chuwy

Common: add support of OWM real-time API

1

Could some subset of OWM calls be done via the real-time API, because the event timestamps are close enough to now() to work? Not sure if this is possible -...

chuwy