Destination databricks: switch to oss jdbc driver

[edgao]

closes https://github.com/airbytehq/airbyte-internal-issues/issues/9120

also it seems like we don't need the databricks sdk at all?

The new driver has a slightly different interface (you can't directly supply a URL, it forces you to supply individual fields/properties). I tried to port over our existing stuff, but removed the transportMode=http and EnableArrow=0 things to see if they're still needed.

Databricks documentation doesn't even describe how to do oauth, it only says how to do PAT (https://docs.gcp.databricks.com/en/integrations/jdbc/oss.html#authenticate-the-driver). I copied our old stuff to the new interfaces naively, but it doesn't work

• DatabricksSQLException: Communication link failure. Failed to connect to server. :https://dbc-6aebf761-f8d6.cloud.databricks.com:443accessToken must be defined
• DatabricksSQLException: Communication link failure. Failed to connect to server. :https://dbc-6aebf761-f8d6.cloud.databricks.com:443Cannot invoke "com.databricks.sdk.core.oauth.OpenIDConnectEndpoints.getTokenEndpoint()" because "jsonResponse" is null).

notable changes in the oss driver:

• timestamps with timezone now have a timezone directly from the driver
• timestamps without timezone have .000 precision
• Inline byte limit exceeded. Statements executed with disposition=INLINE can have a result size of at most 26214400 bytes. Please execute the statement with disposition=EXTERNAL_LINKS if you want to download the full result

which means:

• the destinationhandler can parse directly to an Instant, instead of needing to go through LocalDateTime
• tons of changes in the expected records

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Sep 4, 2024 4:56pm

[edgao]

• #44033 👈
• #44505 : 1 other dependent PR (#44506 )
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @edgao and the rest of your teammates on Graphite

[cgardens]

We are closing this PR due to inactivity. We regularly close PRs that were created more than 6 months ago. If you'd like to keep working on it, please feel free reopen it. We'd be happy to keep working with you.