snowplow-rdb-loader icon indicating copy to clipboard operation
snowplow-rdb-loader copied to clipboard

Kinesis transformer: fails to process event with multiple schema versions

Open dkucharc opened this issue 1 year ago • 0 comments

The transformer client is not able to decode server response containing multiple schema versions.

{
  "schema": "iglu:com.snowplowanalytics.snowplow.badrows/loader_iglu_error/jsonschema/2-0-0",
  "data": {
    "processor": {
      "artifact": "snowplow-transformer-kinesis",
      "version": "5.4.0"
    },
    "failure": [
      {
        "schemaCriterion": "iglu:com.custom/user_context/jsonschema/1-*-*",
        "error": {
          "error": "ResolutionError",
          "lookupHistory": [
            {
              "repository": "Iglu Client Embedded",
              "errors": [
                {
                  "error": "NotFound"
                }
              ],
              "attempts": 1,
              "lastAttempt": "2024-01-05T14:53:40.205Z"
            },
            {
              "repository": "Custom",
              "errors": [
                {
                  "error": "ClientFailure",
                  "message": "Could not decode server response. org.http4s.InvalidMessageBodyFailure: Invalid message body: Could not decode JSON: [\n  \"iglu:com.custom/user_context/jsonschema/1-1-0\",\n  \"iglu:com.custom/user_context/jsonschema/1-0-0\"\n]"
                }
              ],
              "attempts": 8,
              "lastAttempt": "2024-01-05T14:55:44.693Z"
            }
          ]
        }
      }
    ],
    "payload": {...}

Deployed stack:

  • snowplow/iglu-server:0.11.0
  • snowplow/transformer-kinesis:5.4.0

dkucharc avatar Jan 05 '24 16:01 dkucharc