gluon-nlp icon indicating copy to clipboard operation
gluon-nlp copied to clipboard

[Numpy] Update configuration system

Open sxjscience opened this issue 4 years ago • 7 comments

@dmlc/gluon-nlp-team

I had an offline discussion with the GluonCV people, @zhreshold and @Jerryzcn. We decide to use sacred as the default configuration system for both Gluon toolkits.

Currently, in the numpy branch, I'm using yacs. Moving forward to use sacred is not super-difficult. The advantage of sacred is that it is more powerful (support highlighting the changed portion of the default config). What's your opinion on the config system?

sxjscience avatar Jul 16 '20 21:07 sxjscience

I tried it before but ran into issues due to https://github.com/IDSIA/sacred/issues/193. Maybe it has been addressed recently in https://github.com/IDSIA/sacred/issues/610 https://github.com/IDSIA/sacred/pull/623 but I haven't checked in detail. So I'd caution against committing to this change without checking the consequences.

leezu avatar Jul 16 '20 22:07 leezu

@leezu What's your opinion on yacs? This is simpler and easier to hack.

sxjscience avatar Jul 16 '20 22:07 sxjscience

If we check the fvcore/common/config: https://github.com/facebookresearch/fvcore/blob/master/fvcore/common/config.py, we may find that it's based on yacs and adds some other functionalities. Currently, I'm doing it here: https://github.com/dmlc/gluon-nlp/blob/a646c34304c4bde9423468714bd2ff6357cd2091/src/gluonnlp/utils/config.py#L4-L26

sxjscience avatar Jul 16 '20 22:07 sxjscience

@leezu what type of issue did you hit with sacred? I have no preference on the configuration system itself, either sacred or yacs will do the job. Currently I am hidding the magic of sacred under the hood and I am not expecting users to touch the config generator. I've tried yacs for config as well, but is swinging towards sacred simply because I can get rid of laborious argparse setup and can benefit from highlighted options and comments.

example screenshot:

image

example script to launch the job:

from gluoncv.auto.estimators.center_net import ex, CenterNetEstimator

@ex.automain
def main(_config):
    # main is the commandline entry for user w/o coding
    c = CenterNetEstimator(_config, None)
    c.fit()

zhreshold avatar Jul 16 '20 22:07 zhreshold

@zhreshold How about users that import gluoncv / gluonnlp as library and want to modify the config without relying on the auto-generated command line interface? I think there were issues with this before related to the use of global state and "magic", but I'm not sure anymore about the exact issues and maybe it was fixed based in the refactor referenced above.

leezu avatar Jul 16 '20 23:07 leezu

Not sure if sacred has worked around the magics, but for the configs itself is a standard dict which can be modified by user anytime

zhreshold avatar Jul 16 '20 23:07 zhreshold

Thanks to the efforts by @zhreshold , who has created the https://github.com/zhreshold/autocfg project (See https://github.com/zhreshold/autocfg/blob/master/examples/basics.ipynb for a demo). GluonCV and GluonNLP may try to use autocfg as the main configuration toolkit so we can have a unified interface for the Gluon toolkits. @leezu @szha @szhengac @barry-jin

sxjscience avatar Oct 21 '20 00:10 sxjscience