khiops
khiops copied to clipboard
Enable output as multiple file blobs instead of a single large file
It would be highly beneficial if the Khiops could generate output in the form of multiple file blobs rather than consolidating everything into a single enormous file. Handling large datasets as a collection of smaller files (blobs) improves scalability, parallel processing, resilience, and cost-efficiency during analysis and modeling tasks.
Storing data as multiple smaller blobs aligns better with modern data processing workflows, enabling easier updates, partial reads, and distributed analysis.
Please consider adding an option to output data as multiple files instead of a single large file, especially when dealing with multi-terabyte datasets.