ENH: Refactor the configuration.

[tobiasraabe]

Is your feature request related to a problem?

• The configuration is a big dictionary which is created by merging cli options, options in the configuration file and default values in this order of precedence.
  • You'll see get_first_non_none_value all over the place which does merging, validation, conversion.
• Each plugin is allowed to dynamically extend the command line interface and, thus, the configuration.

Describe the solution you'd like

The solution needs to fulfill the following criteria.

• The configuration needs mechanisms for merging, validation and conversion.
• The configuration needs to be dynamically extensible.

API breaking implications

The user api is not affected since the configuration object is never exposed.

Plugins need to be revised.

Candidate solutions

typed-settings

• Provides support for reading configuration from many locations (environment variables, toml files), but not ini!

• Provides functionality to and find configuration files. (allows to remove functionality in pytask)

• [ ] Check how well subcommands are supported

• [ ] Check how well dynamic generation of the settings is supported.

Others

• There are a couple of libraries which provide some of the functionality.
  • https://github.com/samuelcolvin/pydantic
  • https://gitlab.com/sscherfke/typed-settings along with attrs.make_class might be able to create dynamic configurations. But, one would need to provide a loader for ini files. It is based on cattrs and here is an article which argues for cattrs and against pydantic: https://threeofwands.com/why-i-use-attrs-instead-of-pydantic/

[tobiasraabe]

#253 is one step in the right direction to move from unstructured ini formats to pyproject.toml.

[tobiasraabe]

#313 removes everything related to INI configurations. Moving to typed-settings for better validation and less custom code seems manageable and promising now.