openwisp
[config] Merging lists
The way configuration is merged with templates now has a quirk: list elements from different templates are merged with the config.
This is needed for list of objects (eg: interfaces, routes), but it's weird and has caused misunderstanding among users when they try to override list options.
I still haven't come up with an optimal solution.
Let's list the options we have so we can get closer to a solution.
- Items of lists with the same name in different templates are merged, resulting in a list containing all items from all templates. This is the current behaviour.
- If there are lists with the same name in different templates, the items of the list in the template with the highest priority will be the items in the resulting list, overriding the items in the other templates. This has the disadvantage that for adding items to a list, all common items need to be listed in several templates.
- We could add an option/setting to the list in our templates which defines the behaviour of that list if there is a list with the same name in a template with lower priority: add items to that lower priority list or replace items. This makes it possible to choose the behaviour depending on the usecase.
@nemesisdesign What do you think?
@okraits option 3. sounds like a good idea, an "advanced option" which allows to decide if the user wants to add to any previous template or override, to be set on a per template basis and it would act only on previous templates, right?
@okraits option 3. sounds like a good idea, an "advanced option" which allows to decide if the user wants to add to any previous template or override, to be set on a per template basis and it would act only on previous templates, right?
Yes, each template would basically take the result of the previous templates as input and then override that or add to it, based on the selected advanced option.
@okraits how do you intend to implement this? Are we going to add an initialization argument to the library?
You mention adding an "option" to the list, can you expand on this?
@okraits how do you intend to implement this? Are we going to add an initialization argument to the library?
You mention adding an "option" to the list, can you expand on this?
@nemesifier
To be honest, I have to look into the code first to see what's possible. I'll get back to you with a proposal, ok?
@nemesifier I thought about the whole topic and possible solutions.
Originally I thought about adding a "meta setting" to every list in the json configuration dictionary (which would mean that we would have to store a list in json in a surrounding dictionary to be able to have a "meta setting") to define wether the corresponding list should be merged with or override an existing list. But I guess that this is too complex, we would need to extend the json schema and possibly modify all the parsers, converters and renderers.
So I guess the more simple solution is to add a field to the AbstractTemplate model to define the behaviour of lists for each template. I can think of the following behaviours:
- override/replace an existing list
- insert items at the beginning of an existing list
- append items to the end of an existing list
Am I correct that we basically need to modify the following functions to implement the behaviours mentioned above:
- https://github.com/openwisp/netjsonconfig/blob/master/netjsonconfig/utils.py#L6
- https://github.com/openwisp/netjsonconfig/blob/master/netjsonconfig/utils.py#L35
What do you think?
@nemesifier I didn't receive a response from you, so I implemented this. Please take a look at it and feel free to ask questions or request changes.
P.S. I will first wait for your feedback and fix the tests afterwards.