[TECH DEBT]: Add log statements to `crawl_tables` to track listing databases and retriving table information

[JCZuurmond]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

No log statements are visible

Expected Behavior

Like the print statements in the attachment, but with a proper logger setup

tables.scala.txt

Steps To Reproduce

NA

Cloud

Azure

Operating System

macOS

Version

latest via Databricks CLI

Relevant log output

No response

[nfx]

we can rewrite the crawl tables from scala to python, so that the only benefit we get would be the logging. but we have to be careful on the cluster types we use. Pending the Ext HMS design decisions to be made next week.