FuxiCTR icon indicating copy to clipboard operation
FuxiCTR copied to clipboard

Test NVTabular, Petastorm, and Huggingface Datasets for parquet data loading

Open zhujiem opened this issue 10 months ago • 0 comments

Huggingface Datasets:

 dataset = load_dataset("parquet", data_files={split: data_blocks}, split=split)
 super().__init__(dataset=dataset, num_workers=8, batch_size=self.batch_size)

zhujiem avatar Apr 21 '24 14:04 zhujiem