unitxt icon indicating copy to clipboard operation
unitxt copied to clipboard

Add plugin operators

Open elronbandel opened this issue 1 year ago • 3 comments

This is a proposal for a new behaviour that allows to change card operators from the final command such that: load_dataset("card=cards.wikitq,table_serializer=serializers.table.markdown") will be loading the wikitq with different table serializer, markdown serializer in this case.

elronbandel avatar Feb 14 '24 13:02 elronbandel

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (0e2d06e) 87.54% compared to head (05fbded) 87.66%. Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #574      +/-   ##
==========================================
+ Coverage   87.54%   87.66%   +0.12%     
==========================================
  Files          83       84       +1     
  Lines        7104     7143      +39     
==========================================
+ Hits         6219     6262      +43     
+ Misses        885      881       -4     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 14 '24 13:02 codecov[bot]

If I understand there are two changes here.

  1. The ability to add constant fields to the datasets via the recipe (what happens If it overwrites an existing field?)
  2. An operator PlugIn operators that by defaults apply an operator to the stream, but allows changes the operator.

The Plugin operator is a bit tricky to understand (even the example is).

yoavkatz avatar Feb 14 '24 14:02 yoavkatz

A naïve question.. : Since the card is a json file, a simpler, and perhaps better understood way (addressing @yoavkatz concern) would be to first load that json file into a json object, then change that json object, and use the edited version as the input to the recipe? Or the card must be instantiated via fetch_artifact? can that ad-hoc edited card be added to the catalog via add_card?

dafnapension avatar Feb 14 '24 15:02 dafnapension