openpi icon indicating copy to clipboard operation
openpi copied to clipboard

Error when using own local dataset to fine-tune pi05

Open zhouyc1006 opened this issue 4 weeks ago • 1 comments

Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/datasets/builder.py", line 1855, in _prepare_split_single for _, table in generator: File "/.venv/lib/python3.11/site-packages/datasets/packaged_modules/parquet/parquet.py", line 90, in _generate_tables if parquet_fragment.row_groups: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pyarrow/_dataset_parquet.pyx", line 385, in pyarrow._dataset_parquet.ParquetFileFragment.row_groups.__get__ File "pyarrow/_dataset_parquet.pyx", line 392, in pyarrow._dataset_parquet.ParquetFileFragment.metadata.__get__ File "pyarrow/_dataset_parquet.pyx", line 381, in pyarrow._dataset_parquet.ParquetFileFragment.ensure_complete_metadata File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status pyarrow.lib.ArrowInvalid: Could not open Parquet input source '<Buffer>': Parquet file size is 0 bytes

The above exception was the direct cause of the following exception: dataset = lerobot_dataset.LeRobotDataset( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/lerobot/common/datasets/lerobot_dataset.py", line 499, in __init__ self.hf_dataset = self.load_hf_dataset() ^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/lerobot/common/datasets/lerobot_dataset.py", line 620, in load_hf_dataset hf_dataset = load_dataset("parquet", data_dir=path, split="train") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/datasets/load.py", line 2084, in load_dataset builder_instance.download_and_prepare( File "/.venv/lib/python3.11/site-packages/datasets/builder.py", line 925, in download_and_prepare self._download_and_prepare( File "/.venv/lib/python3.11/site-packages/datasets/builder.py", line 1001, in _download_and_prepare self._prepare_split(split_generator, **prepare_split_kwargs) File "/.venv/lib/python3.11/site-packages/datasets/builder.py", line 1742, in _prepare_split for job_id, done, content in self._prepare_split_single( File "/.venv/lib/python3.11/site-packages/datasets/builder.py", line 1898, in _prepare_split_single raise DatasetGenerationError("An error occurred while generating the dataset") from e datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset

Has anyone encountered this problem? Thanks in advance!

zhouyc1006 avatar Nov 06 '25 06:11 zhouyc1006