texar-pytorch
texar-pytorch copied to clipboard
Provide built-in routines from loading config `.py` files
Configurations in examples (e.g. transformer/config_model.py) is written in a separate file with config values as global variables. To be able to use these config files, we import them with importlib
, and later convert them into dictionaries so we can pass it as hparams
. An example here:
https://github.com/asyml/texar-pytorch/blob/aedf40a55dbe324840186e870fc0e414bb94d0c3/examples/bert/bert_classifier_main.py#L56-L59
While this approach works, it certainly has downsides:
- Importing a module from a user-given location is unsafe, as Python executes the code during import.
- As illustrated above, converting an imported module to dictionary is non-trivial (has to ignore magic dunder (
__
) variables).
I'm proposing that we provide built-in mechanisms to deal with importing config modules. This could be a simple wrapper of the above code snippet that imports the the config module and converts it into a dictionary. It should also be able to work with file paths such as configs/config_data.py
(instead of dot notation configs.config_data
).
This would solve the second issue but not the first one, which I think would be impossible to solve as long as we still use config modules. In the future, we might think of alternative ways to store configs while still being robust enough.