blazingsql
blazingsql copied to clipboard
[BUG] change config_options from being a map of strings to a proper type
This has caused countless headaches when as its pretty damn easy to make a typo and never find out/
Here is my design proposal for this:
- The current user facing API for BlazingContext and bc.sql stay the same. A dictionary called config_options.
- Lets take the work that the function
load_config_options_from_env
does and put it into a constructor for a ConfigOptions class. This class would have the following features:
- Set and manage defaults values the same way that
load_config_options_from_env
, which are either the hardcoded defaults, or taking defaults from environment variables - Validate that all the keys coming in from the user are valid and if not, provide appropriate warnings.
- Validate the values, make sure they are in the right range or the right type
- Provide getters and setters on the python side for config_option values that we use or set on the python side
- Provide an outputing function to send the output to the c++ side. This could be the same output we have right now, which is a dictionary, but where the keys and values are encoded strings
- This class implies that all the values in the dictionary will now always be set
- On the c++ side, we should do the following:
- The input dictionary will be converted into a struct. So we would be taking all the keys and values and putting them into a struct, that way, from that moment onwards, the config options are typesafe and we cant misspell the keys.
- get rid of all the default setting, since we dont need it.
- We can now pass around the config_options as a const struct.
Why should we use config options still and not just make them nbamed arguments of Blazing ContexT?
It would be a fairly large API change for users and we will be having lots of config options. Not sure if we want to have that many named arguments. I am not extremely opposed to the idea. But not really a fan either.