Generalization of VO and Setup specific configuration settings

[TaykYoku]

Defining the basic concept of VO and Setup specific configuration settings. Moved from #5262.

RFC for discussion in this context: https://github.com/DIRACGrid/DIRAC/wiki/Generalization-of-VO-and-Setup-specific-configuration-settings

[chaen]

Thanks for the RFC, it is well described. I will comment here, as I don't think I can comment on the RFC directly.

I do appreciate this idea of replicating the whole tree in a sub level for consistency. It does make sense, and it is indeed a cheap way to have the multi VO/setup config everywhere. So it probably is a good idea to do it this way.

BUT, I would advocate to have a new directory in the root of the tree, and not to reuse the Operations section. This allows for two things:

• Makes the transition much smoother from the point of view of the user
• Reduces the confusion in the code, and will simplify the next step, which will be to remove the compatibility layer

Something like

DIRAC/..
Registry/..
Systems/..
WebApp/..
Resources/..

Operations/
../DataManagement
../../myOption = myValue

SpecificSectionWhichNameIsToBeDecided/..
../lhcb
../../Operations
../../../DataManagement
../../../../myOption = mylhcbvalue

Moreover, I would like to come back on the example you take in the RFC: the catalog, for which you lookup similar values in Resources and Operations. This is an oddity, and should not be the case. Normally, the backward compatibility for looking up that particular value in Resources should have long been gone. But that's sort of a detail. It just says how bad we are at removing backward compatibility layers :-)

Finally, from the pure code perspective:

# receives a configuration considering VO and a setup. The following examples will have the same result.
Operations(<vo>, <setup>).getValue('/Resources/FileCatalogs/<catalog name>/AccessType')
Operations(<vo>, <setup>, 'Resources').getValue('FileCatalogs/<catalog name>/AccessType')
Operations(<vo>, <setup>, 'Resources/FileCatalogs').getValue('<catalog name>/AccessType')

In my view, only the very first line is valid. I understand that one can save 0.3 seconds if we have to get 3 parameters from the same directory if we factorize the directory in the constructor, but I don't think it is worth it. We have many parameters that have the same name, and having the full path in the argument is the best way to distinguish them, and allows for easier search in the code. So I would strongly advocate to only have the first form.

One word of caution though. Stating "We reproduce the whole tree, and everything is VO specific" is an extremely strong statement. And I am not convinced that we can realistically stick to it for sections like DIRAC. For example, it would be extremely tricky to have DIRAC/Security/UseServerCertificate be vo/setup specific.

[atsareg]

I agree with Chris that a new section, let say VOSettings, should be introduced and be largely a copy of the current Operations. Also a path given in the Operations.getValue() should always be absolute and not relative to the section in the constructor arguments.

[TaykYoku]

agree also, given that the Operations section contains "its options" such as pilot settings, it is better to create a new section for VO/setup sensitive settings.

[TaykYoku]

I tried to integrate all the ideas in #5331. Currently, these changes do the following:

DIRAC/..
Registry/..
Systems/..
WebApp/..
Resources/..

Operations/
../DataManagement
../../myOption = myValue

SenSettings/..   # sensitive to the VO/setup settings
../lhcb
../../Operations
../../../DataManagement
../../../../myOption = mylhcbvalue

Usage:

1. To get configuration for default VO/setup (or in a singleVO installation), just as usual:

from DIRAC import gConfig

# lhcb is default VO, then result is "mylhcbvalue"
gConfig.getValue('/Operations/DataManagement/myOption')

2. If you need to get configuration for some another VO, you can use gConfig:

gConfig['lhcb'].getValue('/Operations/DataManagement/myOption')

OR

gConfig.getValue('/Operations/DataManagement/myOption', vo='lhcb')

OR create new ConfigurationClient instance

from DIRAC.ConfigurationSystem.private.ConfigurationClient import ConfigurationClient
ConfigurationClient(vo='lhcb').getValue('/Operations/DataManagement/myOption')

3. The same if you need to consider another setup:

gConfig['lhcb', 'mySetup'].getValue('/Operations/DataManagement/myOption')

gConfig.getValue('/Operations/DataManagement/myOption', vo='lhcb', setup='mySetup')

ConfigurationClient(vo='lhcb', setup='mySetup').getValue('/Operations/DataManagement/myOption')

[TaykYoku]

Maybe it's worth tying it to some milestone.. 8.0?

[fstagni]

Given that this is not something for which we will do more development in DIRAC, I close this issue.