EJ Song

@dennyglee @scottsand-db @zsxwing @tdas Could you review the PR? I'll resolve the conflict once you started actively reviewing.

@dennyglee @scottsand-db @zsxwing @tdas @allisonport-db Could you review the PR?

> @sezruby Any news on this PR? We're running into this issue a lot, having a streaming job that writes to the same partition 24/7, and so we are having...

@zedtang Are you planning to support "cluster on write" ?

Seems the fix is needed for all other versions. in MergeIntoCommand ``` override val canMergeSchema: Boolean = conf.getConf(DeltaSQLConf.DELTA_SCHEMA_AUTO_MIGRATE) ``` `conf` is `SQLConf.get` from QueryPlan - LogicalPlan - Command - RunnableCommand

1 => We have logs and I can find the case in spark 3.2/1.2 too, using ``` | where (numTargetCopiedRows + numTargetUpdatedRows) > 0 and numTargetDeletedRows == 0 and numOutputRows...

I confirmed they are using multi threads (python) to run MERGE command to tables concurrently, but not changing the config nor creating a new session for each thread. However, there...

@johanl-db Any delivery plan from Databricks by any chance? I can revert changes for other command if we need to do it with following up PRs + additional tests. I...

> @sezruby I updated the changes against all branches except 1.0 to cover executeDetails, executeRestore, executeGenerate and optimize. Can you update this PR against 1.0 before we merge it? I...

What about 1. Get the list of files for the latest version 2. Copy all the files, using same directory structure (e.g. /path/to/table/A=1/a.parquet should be copied to /path/to/backuptable/A=1/a.parquet) 3. Copy...