transformers icon indicating copy to clipboard operation
transformers copied to clipboard

HfArgumentParser support yaml parser

Open jiangwangyi opened this issue 3 years ago • 1 comments

Feature request

HfArgumentParser now supports for parsing dict and json files, will it be possible to support for parsing the widely used yaml files?

Motivation

I think using yaml is a good way to record arguments.

Your contribution

Not yet.

jiangwangyi avatar Sep 20 '22 07:09 jiangwangyi

cc @sgugger

If you want to open a PR, please go ahead!

LysandreJik avatar Sep 20 '22 19:09 LysandreJik

You can just use parser.parse_dict(yaml.safe_load(f))

felix-schneider avatar Sep 23 '22 08:09 felix-schneider

Which could all go in a parse_yaml_file method :-) Doing this and also refactoring the parse_json_file to use parse_dict, as well as adding small tests would be nice additions that shouldn't be too hard, so putting the "Good first issue" label here.

To summarize:

  • [ ] adding as parse_yaml_file method to HfArgumentParser with the code above
  • [ ] refactor the dupe code between parse_json_file and parse_dict similar to the code above
  • [ ] add a small test of parse_yaml_file
  • [ ] add a small test of parse_json_file

This could be done in a single PR or separate ones :-)

sgugger avatar Sep 23 '22 11:09 sgugger

Hi, I would like to work on it

IMvision12 avatar Sep 24 '22 20:09 IMvision12

How can i write test for parse_yaml_file and parse_json_file it will require an external json and yaml file to testing

IMvision12 avatar Sep 27 '22 07:09 IMvision12

No, you can create it during the test by saving some dictionary (look at the parse_dict tests) into a temporary file.

sgugger avatar Sep 27 '22 11:09 sgugger

Hey, @sgugger I have written the test for parse_yaml_file and parse_json_file using tempfile is it acceptable?? Also it passes the tests. Capture

IMvision12 avatar Sep 27 '22 13:09 IMvision12

You can also use the context manager for a temp dir.

with tempfile.TemporaryDirectory() as tmp_dir:
    # Save file in tmp_dir as usual
    # do the tests

The plus for this is that it's automatically cleaned up when you exit the with block (whereas the temp file will stay until the next restart).

sgugger avatar Sep 27 '22 14:09 sgugger

Okay I will change that!

IMvision12 avatar Sep 27 '22 15:09 IMvision12