unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Allow using python functions instead of operators (e.g in pre-processing pipeline)

Open elronbandel opened this issue 6 months ago • 0 comments

If you cannot find operators fit to your needs simply use function to modify every instance in the data:

        def my_function(instance, stream_name=None):
            instance["x"] += 42
            return instance

Or a function that modify the entire stream:


        def my_other_function(stream, stream_name=None):
            for instance in stream:
                instance["x"] += 42
                yield instance

Both functions can be plugged in every place in unitxt requires operators, e.g pre-processing pipeline.

elronbandel avatar Jun 26 '25 14:06 elronbandel