datatrove icon indicating copy to clipboard operation
datatrove copied to clipboard

Spark support

Open jordane95 opened this issue 1 year ago • 3 comments

I'm wondering if it is possible to add support for other popular large-scale data processing frameworks like spark, since most operations are compatible with the map operation in spark. This would greatly improve the efficiency and scability of the processing pipeline when working with large datasets.

jordane95 avatar Jan 30 '24 15:01 jordane95