Daniel Tomes

Also closely related to #395

Assuming this only partially resolves #311 since we're likely still needing to get the instanceProfile items to Gold, right?

@Sriram-databricks -- please approve this PR. I opened it (for some silly reason) so I cannot approve it. LGTM

depends on #503 -- once 503 is completed this can be added to min schema but not before as it is an array of struct

right, but there is a limitation on Event hubs per namespace i believe (or per subscription). After investigation, I've determined that it's possible to do this but it would require...

Adding to 6.1.3 for feasibility review

@alexott -- I have created this filter string to enable this -- but I'm now concerned about the amount of data scanned and thrown away increasing runtimes and/or costs (EH...

Note that the following also exists. I'm not sure if its pattern is consistent but it's something to look into as we look further into this ticket. `aott-db` is the...

@Sriram-databricks -- let's review perf differences in this and see if it makes sense (P1) -- if we cannot get it into 0.6.1.2 that's ok

Will not use EH for this -- Investigate Kafka enabled EH can improve this