otoole icon indicating copy to clipboard operation
otoole copied to clipboard

Parameters with more than one index (e.g. region, region) cause a read error

Open willu47 opened this issue 1 year ago • 2 comments

If a config contains a parameter which contains a duplicate index, such as

TradeRoute:
    indices: [REGION,REGION,FUEL,YEAR]
    type: param
    dtype: float
    default: 0

then an error is raised when reading in the corresponding csv file

NotImplementedError                       Traceback (most recent call last)
Cell In[2], line 24
     20 validate_config(config)
     22 read_strategy = ReadCsv(user_config=config)
---> 24 model, defaults = read_strategy.read(folder_path)
     25 logging.debug(model.keys())

File ~/miniconda3/envs/linopy/lib/python3.11/site-packages/otoole/read_strategies.py:209, in ReadCsv.read(self, filepath, **kwargs)
    207 if entity_type == "param":
    208     df = self._get_input_data(filepath, parameter, details, converter)
--> 209     narrow = self._check_parameter(df, details["indices"], parameter)
    210     if not narrow.empty:
    211         narrow_checked = check_datatypes(
    212             narrow, self.user_config, parameter
    213         )

File ~/miniconda3/envs/linopy/lib/python3.11/site-packages/otoole/read_strategies.py:91, in _ReadTabular._check_parameter(self, df, expected_headers, name)
     87         logger.warning("%s not in header of %s", column, name)
     89 logger.debug("Final all headers for %s: %s", name, all_headers)
---> 91 return narrow[all_headers].set_index(expected_headers)

File ~/miniconda3/envs/linopy/lib/python3.11/site-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments..decorate..wrapper(*args, **kwargs)
    325 if len(args) > num_allow_args:
    326     warnings.warn(
...
    420     values = sanitize_array(values, None)
    421 else:
    422     # i.e. must be a list

NotImplementedError: > 1 ndim Categorical are not supported at this time

I'm using pandas v1.5.3 and otoole v1.0

willu47 avatar Apr 10 '23 11:04 willu47