hadoop-lakefs integration tests to run over Databricks

[nopcoder]

Currently, we run an integration test using docker on Spark cluster 2.x and 3.x. Running the same on Databricks will help us catch more cases and issues before publishing a new package.

[johnnyaug]

In our integration tests we start lakeFS in a docker-compose in GitHub actions. If we use the same way here, we need to somehow give Databricks access to this installation. I tried to solve this with an ngrok tunnel, but in the end found out that they have a limit of 1 online process at a time (2 in the Business plan) so it's not a suitable solution for us.

[github-actions[bot]]

This issue is now marked as stale after 90 days of inactivity, and will be closed soon. To keep it, mark it with the "no stale" label.

[github-actions[bot]]

Closing this issue because it has been stale for 7 days with no activity.