Fix logical vs physical schema mismatch for right side field schema selection.

[wiedld]

Which issue does this PR close?

Part of #12733 closes https://github.com/apache/datafusion/issues/13010

Rationale for this change

We found another bug for when the logical vs physical schema does not match. In this specific case, the UNION schema will select which side has the nullable field -- and default to taking the left side.

In the example case, we have the nullable field as the right side. With this setup, it became evident that we are not adding the field metadata from the left side => to the right side.

What changes are included in this PR?

First commit is the reproducer. Second commit is the fix. Third commit updates the test/reproducer now that the fix is in.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

[alamb]

Marking as draft as I think this PR is no longer waiting on feedback. Please mark it as ready for review when it is ready for another look

[alamb]

I filed https://github.com/apache/datafusion/issues/13010 specifically to track this issue so we don't lose track of it

[wiedld]

Marking as draft as I think this PR is no longer waiting on feedback. Please mark it as ready for review when it is ready for another look

Thank you. As you know, I've been shifted to a higher priority item. 🙏🏼

[itsjunetime]

I took this over on our end and just pushed some more commits to

1. Fix another issue I ran into (specifically, when unioning multiple schemas where the first is not the one with the metadata)
2. Make it work with multiple schemas

I'll rebase it soon and I can add some more comments to fix the concerns commented here earlier.

[alamb]

Thanks again @itsjunetime and @wiedld