hf-datasets topic

An extension package of 🤗 Datasets that provides support for executing arbitrary SQL queries on HF datasets

mariosasko

This project shows how to derive the total number of training tokens from a large text dataset from 🤗 datasets with Apache Beam and Dataflow.

sayakpaul