Kishor Radhakrishnan

[log4j-active (7).txt](https://github.com/microsoft/Purview-ADB-Lineage-Solution-Accelerator/files/11166757/log4j-active.7.txt)

@hmoazam Yes, As this error is different and we are seeing it frequent in DBR 10.4 LTS 23/04/05 08:27:02 ERROR EventEmitter: Could not emit lineage w/ exception java.net.SocketTimeoutException: Read timed...

@wjohnson attaching log having this error from monitor [openlineage.txt.zip](https://github.com/microsoft/Purview-ADB-Lineage-Solution-Accelerator/files/11340865/openlineage.txt.zip)

@wjohnson I am adding a snippet of code where write occur. `def write_dataframe(df,tgt_path,part_col): try: if part_col != "NA": ( df .write .format("delta") .mode("overwrite") .partitionBy(part_col) .option("mergeSchema","true") .option("overwriteSchema","true") .save(tgt_path) ) print("Data successfully...

also we face this issue for many different notebooks may be . we need to see why partition is not identified properly. May we can try short term solution of...

@wjohnson I tried setting with much lower value . Still we are seeing failures. Set value as 10000

Tried the same . Still we are seeing many errors

Yes, am seeing that in logs . But still it looks many events are exceeding eventhub limits


[query_data.csv.zip](https://github.com/microsoft/Purview-ADB-Lineage-Solution-Accelerator/files/11250806/query_data.csv.zip) Latest exception logs