Chester Chen

Thanks for looking at. The sbt-idea plugin is great. Saved me a lot of times. The project setup came from following consideration. I want to enforce the clear separation for...

I created a sample multi project setup in https://github.com/chesterxgchen/multiproject you can run sbt gen-idea on the parent project to see the issue.

@mengxr I am also wondering how this be integrated with other pipelines such as Airflow, as part of the pipelines are using Scala-based spark jobs do the heavy lifting transformation...

In above scenarios, the difference from the original scripts is that the delight.jar is not copied to /mnt/xxx directory ( but installed directly in databricks cluster library (via cluster API),...

I would be interesting to see this as well. Like to create a config file, for each config add comment to certain section programly.

In data management, should include new data formats such as Apache Iceberg -- originally from Netflix, currently used by many big companies ( Netflix, Apple, Alibaba, Tencent, Adobe, LinkedIn (?),...

Monitoring -- Data quality monitoring : features quality, feature distribution visualization, feature skew, data distribution change over time. Feature training/test/validation distributions mismatch etc Model monitoring --