transformers
transformers copied to clipboard
HfArgumentParser support yaml parser
Feature request
HfArgumentParser now supports for parsing dict and json files, will it be possible to support for parsing the widely used yaml files?
Motivation
I think using yaml is a good way to record arguments.
Your contribution
Not yet.
cc @sgugger
If you want to open a PR, please go ahead!
You can just use
parser.parse_dict(yaml.safe_load(f))
Which could all go in a parse_yaml_file method :-) Doing this and also refactoring the parse_json_file to use parse_dict, as well as adding small tests would be nice additions that shouldn't be too hard, so putting the "Good first issue" label here.
To summarize:
- [ ] adding as
parse_yaml_filemethod toHfArgumentParserwith the code above - [ ] refactor the dupe code between
parse_json_fileandparse_dictsimilar to the code above - [ ] add a small test of
parse_yaml_file - [ ] add a small test of
parse_json_file
This could be done in a single PR or separate ones :-)
Hi, I would like to work on it
How can i write test for parse_yaml_file and parse_json_file it will require an external json and yaml file to testing
No, you can create it during the test by saving some dictionary (look at the parse_dict tests) into a temporary file.
Hey, @sgugger I have written the test for parse_yaml_file and parse_json_file using tempfile is it acceptable?? Also it passes the tests.
You can also use the context manager for a temp dir.
with tempfile.TemporaryDirectory() as tmp_dir:
# Save file in tmp_dir as usual
# do the tests
The plus for this is that it's automatically cleaned up when you exit the with block (whereas the temp file will stay until the next restart).
Okay I will change that!