fairseq2 icon indicating copy to clipboard operation
fairseq2 copied to clipboard

using pipeline_builder shared pointer multiple times lead to segfaults

Open artemru opened this issue 3 months ago • 1 comments

Describe the bug: Segfault during the pipeline creation

Describe how to reproduce:

from fairseq2.data import read_sequence
from fairseq2.data.data_pipeline import DataPipeline, DataPipelineBuilder

pipeline_build = read_sequence(list(range(100)))  # this's shared for two shuffling operations

concat_pipe = DataPipeline.concat([pipeline_build.shuffle(10).and_return(),
                                   pipeline_build.shuffle(10).and_return()]).and_return()
# this should have raised 
next(iter(concat_pipe))

Describe the expected behavior: This should raise an explicit RuntimeError

artemru avatar Mar 05 '24 22:03 artemru