chronon
chronon copied to clipboard
airbnb
Chronon is a data platform for serving for AI/ML applications.
Hey, I read throught the README and had these thoughts (Mostly thoughts rather than corrections) @ezvz ## Summary ## Why / Goal ## Test Plan - [ ] Added Unit...
## Summary Changing all printlns to user a logger. ## Why / Goal Cleaning up. ## Test Plan - [x] Local build, local run, unit tests ## Reviewers @nikhilsimha
## Summary Improve the scalability of the join between the LHS and GroupBys by breaking up the join. Previously, when joining together a large number of GroupBys, the Spark job...
## Summary Added a new flag to overwrite metadata if needed to avoid dropping previously computed intermediate tables ## Why / Goal This PR can avoid dropping the previously computed...
## Summary The group by upload input rdd has less number of partitions with compact size. It can leads to executor OOM while converting to chronon row. Use the default...
## Summary Filters out contextual features from external parts from the comparison dataframe before computing consistency metrics with the logged data. ## Why / Goal The logged data from the...
## Summary Setting a time limit of 40 minutes for circle CI as metadata calls can run into a 4hr+ loop of failure Example: https://app.circleci.com/pipelines/github/airbnb/chronon/3408/workflows/2fdc3dd8-904b-4d90-b062-d5879f4bd686/jobs/17449 From: https://support.circleci.com/hc/en-us/articles/11775183948827-How-to-run-a-job-in-the-background-on-CircleCI > A job...
## Summary Two sample notebooks developed when experimenting with user experience. * One depends on polynote having access to scala and python. * Another relies on mutating some fields, writing...
## Summary From the new release, we are seeing ``` Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.sql.catalyst.util.RebaseDateTime$ ``` https://airflow-1-10.d.musta.ch/log?task_id=compute_join__fiv__target_features_listing__v1&dag_id=chronon_join_fiv__target_features_listing__v1&execution_date=2023-10-18T00%3A00%3A00%2B00%3A00 Adding this class to see if it can...
## Summary WIP ## Checklist - [ ] Documentation update ## Reviewers @hzding621
