Simplify the input database structure

[datejada]

Discussed in https://github.com/spine-tools/SpineOpt.jl/discussions/581

^{Originally posted by nhniina March 24, 2023} The number of classes and parameters in the input database structure feels a bit overwhelming for many new users. Particularly, there seem to be very many classes related to the temporal and stochastic structure. Are there ways to simplify or clarify the structure without reducing the functionalities or making it more difficult to use the advanced features of SpineOpt?

One option could be to categorise the classes in the DB Editor. This would probably require additional features to the DB Editor itself. Some classes are there to introduce certain variables whereas some classes are "just" to store parameters. However, it is not easy to grasp that idea currently. In addition, there are multiple classes related to advanced temporal and stochastic structures as well as user constraints. Thus, there could be categories, for example, for

• parameters
• variables
• advanced temporal and stochastic structures
• user constraints

I suppose there are cases where a class belongs to multiple categories. How to handle those cases?

Ideas for the classes related to the temporal and stochastic structure

1. Create a new class group to group connections, nodes and units with similar temporal and stochastic structures. Create also new classes connection__group, node__group and unit__group. Replace corresponding connection/node/unit__... classes with

• group__investment_stochastic_structure
• group__stochastic_structure
• group__investment_temporal_block
• group__temporal_block

Perhaps in many cases the user wants to create a specific temporal structure for all biomass nodes or all connections, nodes and units in a neighbouring country. The new class group could make these kind of cases easier. However, there are also cases when the user wants to create a specific structure for just one node, for example. In those cases the additional layer can decrease usability.

2. Create a new class type (or something similar) with fixed entities investment, flow, on (or something similar, corresponding to the variable names). Then, replace classes ...__investment_stochastic_structure, ...__stochastic_structure, ...__investment_temporal_block, ...__temporal_block with new classes ...__type__stochastic_structure and ...__type__temporal_block.

However, we should somehow be able to indicate for the user which are valid combinations. This can make the structure just more complicated to understand.

3. Similar to 1. but create the group class to group temporal blocks and stochastic structures instead of connections, nodes and units. For that purpose, create also classes stochastic_structure__group and temporal_block__group. Replace corresponding ...temporal_block/stochastic_structure classes with

• connection__investment_group
• model__default_investment_group
• model__default_group
• model__group
• node__investment_group
• node__group
• unit__investment_group
• unit__group

4. Replace temporal_block and stochastic_structure classes totally with stochastotemporal_structure class, perhaps with a parameter type and parameter value list [temporal data, stochastic data].

5. Use Map parameters instead of temporal and stochastic classes. For example, create a model class parameter temporal_blocks with contents block01, block02, block03. Where to give the necessary data for the blocks? In the same (multi-dimensional/nested) Map or elsewhere? How to make sure that the user gives consistent data? Using upcoming features of Toolbox?

6. Could/should we change stochastic_scenario and stochastic_structure__stochastic_scenario classes somehow?

Ideas for other classes

• To/from nodes: Should we change the classes with to_node and from_node? Some would prefer having direction__node, that is, more dimensions in the classes. Some would prefer to use input and output instead of to and from.
• User constraints: Is there a way to clarify the user constraint classes? Can they somehow be combined with the group class (that groups connections, nodes and units or temporal blocks and stochastic structures; options 1. and 3. above)?
• Reports and outputs: In the input database, the additional layer of report in defining how the outputs are written can feel cumbersome. Is there a way to use entity groups for that? Maybe not?
• Remove unit__commodity class?

[clizbe]

@nhniina Can you please update the description with the outcomes of the discussion? That'll make it more clear what the programmer needs to do. :)

[nhniina]

The decision was to first implement a database upgrade tool here https://github.com/spine-tools/SpineOptPlugin/tree/datastructure_update, and once that works, implement the required changes in SpineOpt code. Most of the planned updates are already implemented in the database upgrade tool. The remaining conversions are related to unit flow ratios (efficiencies) and connections.

[nhniina]

I've added a few tables about the changes here: https://github.com/spine-tools/SpineOpt.jl/discussions/1101