datasets icon indicating copy to clipboard operation
datasets copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Support skipping Parquet to Arrow conversion when using Beam

Open albertvillanova opened this issue 2 years ago • 2 comments

albertvillanova avatar Jun 16 '22 08:06 albertvillanova

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

HuggingFaceDocBuilderDev avatar Jun 16 '22 08:06 HuggingFaceDocBuilderDev

When #4724 is merged, we can just pass file_format="parquet" to download_and_prepare and it will output parquet fiels without converting to arrow

lhoestq avatar Jul 27 '22 16:07 lhoestq

I think we can close this one

lhoestq avatar Nov 07 '22 16:11 lhoestq