blazingsql icon indicating copy to clipboard operation
blazingsql copied to clipboard

[BUG] change config_options from being a map of strings to a proper type

Open felipeblazing opened this issue 3 years ago • 3 comments

This has caused countless headaches when as its pretty damn easy to make a typo and never find out/

felipeblazing avatar Sep 21 '20 14:09 felipeblazing

Here is my design proposal for this:

  1. The current user facing API for BlazingContext and bc.sql stay the same. A dictionary called config_options.
  2. Lets take the work that the function load_config_options_from_env does and put it into a constructor for a ConfigOptions class. This class would have the following features:
  • Set and manage defaults values the same way that load_config_options_from_env, which are either the hardcoded defaults, or taking defaults from environment variables
  • Validate that all the keys coming in from the user are valid and if not, provide appropriate warnings.
  • Validate the values, make sure they are in the right range or the right type
  • Provide getters and setters on the python side for config_option values that we use or set on the python side
  • Provide an outputing function to send the output to the c++ side. This could be the same output we have right now, which is a dictionary, but where the keys and values are encoded strings
  • This class implies that all the values in the dictionary will now always be set
  1. On the c++ side, we should do the following:
  • The input dictionary will be converted into a struct. So we would be taking all the keys and values and putting them into a struct, that way, from that moment onwards, the config options are typesafe and we cant misspell the keys.
  • get rid of all the default setting, since we dont need it.
  • We can now pass around the config_options as a const struct.

wmalpica avatar Oct 28 '20 22:10 wmalpica

Why should we use config options still and not just make them nbamed arguments of Blazing ContexT?

felipeblazing avatar Oct 28 '20 22:10 felipeblazing

It would be a fairly large API change for users and we will be having lots of config options. Not sure if we want to have that many named arguments. I am not extremely opposed to the idea. But not really a fan either.

wmalpica avatar Oct 28 '20 22:10 wmalpica