snowplow-rdb-loader icon indicating copy to clipboard operation
snowplow-rdb-loader copied to clipboard

Published 20 hours ago • verified publisher icon snowplow

RDB Shredder: explore a way to reject pipeline's auxiliary payloads

Open chuwy opened this issue 6 years ago • 1 comments

One of our users mistakenly sent a com.snowplowanalytics.snowplow/contexts payload and Loader failed with following error:

Data discovery error with following issues:
JSONPath file [com.snowplowanalytics.snowplow/contexts_1.json] was not found

Although, this is an expected behavior, it also means that anyone can stop a pipeline by sending this particular (or another similar) entity.

chuwy avatar Mar 04 '19 16:03 chuwy

I think in the end, this (https://github.com/snowplow/snowplow-rdb-loader/issues/59) should be a proper solution that doesn't involve a special treatment for arbitrary schemas.

chuwy avatar Mar 04 '19 18:03 chuwy