Johan Lasperas

I created a design doc to implement support for WHEN NOT MATCHED BY SOURCE clauses: [[Design Doc] WHEN NOT MATCHED BY SOURCE](https://docs.google.com/document/d/13DRQNu3k0ndUIvzGnNnyyiF9e5sQR1NurFVJxbhGgXY/edit?usp=sharing). This enables selectively updating or deleting target rows...

I actually considered this when implementing non-star schema evolution, the main concern was that schema evolution then becomes very error-prone as any typo will happily add a new column to...

Schema evolution allows adding columns/fields that are present in the source dataframe but missing in the table schema, the use case you are describing is different though since the field...

User-facing change 1: Assigning to a column that isn't in the target or source now adds that column to the target table instead of failing: ``` target: key int source:...

The order of the clauses shouldn't have an impact on the resulting type of a column, that would be arbitrary. This rules out 1. and 3. and leaves 2., the...

Thanks @sezruby for contributing a fix. A few questions regarding the issue to help me understand better what's happening: 1. Are you able to reproduce it on other Delta versions...

Rephrasing your findings @clee704, let me know if you disagree: So the issue is essentially that in [DeltaMergeBuilder.execute()](https://github.com/delta-io/delta/blob/5f9b98e86590cf8891c73eb2b78482aee89547d1/spark/src/main/scala/io/delta/tables/DeltaMergeBuilder.scala#L267), we use the spark session from the target table that gets propagated...

I created cherry-picks to all 2.x branches and master: - 3.0: https://github.com/delta-io/delta/pull/2156 - 2.4: https://github.com/delta-io/delta/pull/2157 - 2.3: https://github.com/delta-io/delta/pull/2158 - 2.2: https://github.com/delta-io/delta/pull/2159 - 2.1: https://github.com/delta-io/delta/pull/2160 - 2.0: https://github.com/delta-io/delta/pull/2161 - master: https://github.com/delta-io/delta/pull/2162...

@sezruby I updated the changes against all branches except 1.0 to cover executeDetails, executeRestore, executeGenerate and optimize. Can you update this PR against 1.0 before we merge it?

> > This is because a new thread is created in the JVM process and that thread doesn't inherit any ThreadLocal variables, including the active session ref. > > @johanl-db,...