overwatch icon indicating copy to clipboard operation
overwatch copied to clipboard

Published 20 hours ago • verified publisher icon databrickslabs

[BUG] - Spark_Events_Bronze - taskEndReason.StackTrace

Open GeekSheikh opened this issue 2 years ago • 1 comments

When there are no spark events with a task failure the default return type for taskEndReason.StackTrace is String as it's implicitly cast to string in the ingest dataset since all are null string. This needs to be forced to a struct to ensure it can be merged even when all values are null.

GeekSheikh avatar Jul 29 '22 23:07 GeekSheikh

depends on #503 -- once 503 is completed this can be added to min schema but not before as it is an array of struct

GeekSheikh avatar Sep 23 '22 12:09 GeekSheikh

merged with #649

GeekSheikh avatar Dec 19 '22 16:12 GeekSheikh