[Spark] Add more Delta Connect Python Client integration checks

[longvu-db]

trafficstars

Which Delta project/connector is this regarding?

• [X] Spark
• [ ] Standalone
• [ ] Flink
• [ ] Kernel
• [ ] Other (fill in here)

Description

Following previous PR (https://github.com/delta-io/delta/commit/b7ff92aaaa25e728a8cefae2cd55b07375334533), adding more Delta Connect Python Client integration checks, for QA-ing Delta Connect with Spark Connect 4.0.

Note:

• The goal is mainly to check that the CUJ of starting a local server and client works fine (https://docs.delta.io/3.2.1/delta-spark-connect.html), we don't need to test all of Delta Connect Python Client APIs, which already had UTs.
• This test is similar to https://github.com/delta-io/delta/blob/master/examples/python/quickstart.py
• As discussed, it is not blocker for this release by simplifying the integration test CUJ (in the file's comment at the beginning) to add a knob to run-integration-tests.py like this UC Integration test (https://github.com/delta-io/delta/commit/e23a330021f8056afcf21c7db0ed236f12429b73#diff-5c52e7ef4a1209ed03960987a30691598062d3830b9e86e0c44c3c866bb1c2e1R12-R36), the trouble here, which we already ran into before, is to figure out how we can get the sbin/start-connect-server.sh (downloading the heavy spark 4.0.0 package just to get a bash file is not ideal), this can be done in a future PR if needed.

How was this patch tested?

Rebase with https://github.com/allisonport-db/delta/pull/25 to generate Delta Connect rc2 packages, and locally use the instructions in the comment.

Also did manual QA with the Spark Connect REPL.

Does this PR introduce any user-facing changes?

No.