model-remediation
model-remediation copied to clipboard
TF Dataset to TF Examples List in FDW Utils: Implementation is slow and consumes excessive RAM
The current implementation of TF Dataset to TF Examples List uses a lot of RAM and runs very slowly on large datasets.
In this colab, the CelebA dataset, as processed by the fairness indicators tutorial, can't be processed on a default hosted runtime. After 18-20 minutes, the runtime crashes because RAM requirements are exceeded.
Because of all the preprocessing required, it's not easy to create a smaller reproducible example.