Manu Zhang

Yes, please create a jira issue and open an PR if you've got a solution.

Hi Darion, We previously discussed about time interval rotation at https://github.com/gearpump/gearpump/pull/1260 and the conclusion we only keep file size rotation unless other rotations are required by some use cases.

@zapletal-martin Thanks for your contribution. I'll pull your branch and try playing with it.

Blocked on https://github.com/json4s/json4s/issues/397

Yes, the package name is too long although I don't have a good idea for a shorter one. Maybe `daria._` is enough ?

@huaxingao any update from Spark side?

@rdblue @aokolnychyi @kbendick Please help review and suggest where to add a UT. I haven't found a proper place.

@rdblue I've moved cloning session and disabling AQE into `RewriteDataFilesSparkAction`. When multiple rewrite actions are submitted concurrently as follows, they will block each other when loading table due to locks...

There is another lock in `SessionCatalog#tableExists` for Spark versions before https://github.com/apache/spark/pull/31891 since sessionCatalog is also shared.

> Is that what was happening before? Do you mean session was created for the entire rewrite before?