Will Johnson

@Kishor-Radhakrishnan thank you for your patience! These last logs helped us identify an error in the OpenLineageIn code that removed the spark plan in one variable but failed to remove...

This will be fixed in the next release where we will remove the spark plan and then column lineage information if the payload is still larger than the 1 MB...

@yxu1183 did you attempt to run a notebook and it failed to produce lineage? If you're seeing the `AsyncEventQueue: Process of event SparkListenerApplicationStart` you should be able to receive lineage...

@HBsunflower I know this issue is ancient at this point but do you have any additional details? This should only occur if the OpenLineage payload is malformed as it will...

@gerson23 thank you so much for this great suggestion! I would love to be able to cover Delta Live Tables but we need OpenLineage to support it. In OpenLineage/OpenLineage#372 there...

Adding additional logging for all token and making very clear when a token is being refreshed.

Closing this out as it looks to be complete and no further updates for over a month. Please feel free to open again if there are additional questions.

I am having the same issue as @RothNRK. I've created a pipeline with a DatabricksStep and I am unable to use Run.get_context() to even instantiate the run. I'm using the...

#TakenOffline

Adding additional logs to help with debugging this (as in #225)