unitxt
unitxt copied to clipboard
Allow using python functions instead of operators (e.g in pre-processing pipeline)
If you cannot find operators fit to your needs simply use function to modify every instance in the data:
def my_function(instance, stream_name=None):
instance["x"] += 42
return instance
Or a function that modify the entire stream:
def my_other_function(stream, stream_name=None):
for instance in stream:
instance["x"] += 42
yield instance
Both functions can be plugged in every place in unitxt requires operators, e.g pre-processing pipeline.