Zach
Zach
**Is your feature request related to a problem? Please describe.** Delta Lake collects detailed statistics about each executed transaction in a "history". These statistics should be logged as metrics by...
**Is your feature request related to a problem? Please describe.** Observe custom metrics for executed Spark queries, e.g. Number of Records with special flag, Number of new version in HistorizeAction......
**Is your feature request related to a problem? Please describe.** Ingesting on-premise data sources to a cloud analytics plattform has challenges like - network / firewall security: if there is...
It is a common use case of file transformation to split a big input file into multiple output files. To support this CustomFileAction should support multiple outputs (currently it is...
It should be possible to define additional spark-options for individual Actions and Connections. They should be applied only when executing this Action or an Action which uses a Data Object...
It would be useful if sending an email on success/failure can be configured by additional SmartDataLakeBuilder command line parameters.
As a data scientist i would like to have an Action to apply a ML model inside my data pipeline. **Describe the solution you'd like** New MLFlowAction which applies a...
**Is your feature request related to a problem? Please describe.** An existing Kafka topic with avro schema in schema registry might need a schema evolution if used as output and...
**Is your feature request related to a problem? Please describe.** For initial loads with Deduplicate/HistorizeAction, it might be needed to override reference timestamp, in order to influence first dl_ts_captured column...
**Is your feature request related to a problem? Please describe.** To find which properties an Action or DataObject supports, i find myself often checking scaladoc and code. It would be...