at-python-template
at-python-template copied to clipboard
Reason behind two separate locations for config files?
Depending on the choice of config file type (yaml / hocon / none), different locations for the respecitve config files are used. The hocon choice uses
{{cookiecutter.project_slug}}/src/{{cookiecutter.module_name}}/res/default.conf
while the yaml choice uses
{{cookiecutter.project_slug}}/config/config.yml
The hocon choice has the benefit that this path lies within the sourcepath of the module, meaning that functions like {{ cookiecutter.module_name }}.util.get_resource_string
can fetch config files regardless of module location / installation / etc.
I suggest moving the config.yml
also to the res/
directory and while we're at it, also specifying the default path to the config.yml
in util.load_config
, as it is the case when choosing the hocon option.
The part about the yaml config is correct, but the hocon config setup is more complex. There are two places where hocon config is stored:
-
/src/{{cookiecutter.module_name}}/res/default.conf
: config that ships with the module which contains default values for everything that should have a default value -
/config/debug.conf
: another config that is not shipped with the module that overrides default values - you can have several of those, depending on your use case
A setup like this is only possible, because pyhocon, the hocon parsing library, allows for inheritance between different config files (see util.py
for the implementation). As far as I know, a similar option does not exist when dealing with yaml files.
The purpose of a config file is to provide settings to an application that can easily be changed without editing the source code and recompiling the application. Therefore the config must be a file that is stored outside of your module. The default hocon config is an exception to that rule, because you only provide the defaults in there and still have the option to specify a separate config file that overrides these defaults.
Therefore I believe moving the config.yml
into the module is not a good idea, as long as we can't have a separate config file that can override it.
We could implement a setup which uses a bundled yaml config that can be replaced by some external config, but I don't like that it would completely replace the bundled config. Also, so far noone requested that feature and we should try to avoid adding too much complexity to this template.
Okay, flawless reasoning! :D
So this also implies that, when a package developed with this template is shipped through, let's say, pip, only things below the src/
directory would be contained in that bundle. Thus, no pre-set or template config.yml
is provided to a user for reference or "easily changing config", which is of course the intended behaviour but for non-developer users might then pose a challenge to get proper logging feedback from the package.
But I completely agree that this is nothing to act on proactively, as creating our own yaml-inheritance would needlessly increase the complexity of this template which (as you said) should be avoided because then people might be less inclined to use the template in the first place!
Thanks again for the clarification!!
So this also implies that, when a package developed with this template is shipped through, let's say, pip, only things below the src/ directory would be contained in that bundle
that is correct
Thus, no pre-set or template config.yml is provided to a user for reference or "easily changing config"
well, installing a Python package means that you should not change the source of that package, so there would be no use in shipping a config like that. If you want the user to make use of some kind of config, you should teach them in your documentation on how to do that.
Thanks again for the clarification!!
Thanks for the question! Maybe we should extend the Readme to clarify the usage of the hocon config.
That's a good idea, but a link to the hocon project should suffice. The docstring in the load_config method explains it well enough.
A good addition to the provided default config at both locations could be to showcase the supported variable substitution, because it's quite useful in my opinion. It's actually why I'm only using hocon configs now. It's a great way to set up config files for different environments/stages, like dev.conf, prod.conf, but only set e.g. a variable env=dev
, env=prod
, and the rest of the configuration parameters will get filled automagically substituting the definitions in the default.conf.
@rauerhans I like your suggestion, the distinction between dev and prod would probably be a better way to showcase the usefulness of this feature than the simple debug.conf file we have right now. Would you like to implement it?