Access sub-ingredient's config

[mrzv]

I have two ingredients. One of them knows about the other one. When configuring the second one, I'd like to be able to access a config value from the first one, but I can't find a way to do this. Here's an example that sketches what I'm after.

data = Ingredient('data')
model = Ingredient('model', ingredients = [data])

@data.config
def data_config():
    a = 5

@model.config
def model_config(data):
    b = data['a'] / 2      # this doesn't work

Is there any way to do this?

[mrzv]

Actually, this works as intended; not sure why I thought it didn't. I think I'm actually after "sibling" config. I.e.,

ex = Experiment()

@ex.config
def a_config():
    a = 5


@ex.config
def b_config():
    b = a / 2      # how to make this work?

How can I make something like this work?

[mrzv]

The first version also doesn't work, if I switch model_config to a named_config, i.e.,

@model.named_config
def model_config(data):
    b = data['a'] / 2      # this doesn't work

Then if I invoked ... with model.model_config, I get

Traceback (most recent calls):
  File ".../lib/python3.11/site-packages/sacred/config/custom_containers.py", line 79, in __getitem__
    raise KeyError(item)
KeyError: 'a'

[mrzv]

In the second example (sibling config), it should also be named_config:

ex = Experiment()

@ex.config
def a_config():
    a = 5

@ex.named_config
def b_config(a):
    b = a / 2      # how to make this work?

I get

Traceback (most recent calls):
  File ".../lib/python3.11/site-packages/sacred/config/config_scope.py", line 64, in __call__
    raise KeyError(
KeyError: "'a' not in preset for ConfigScope. Available options are: {'_log'}"

So this is really a question about named_configs.

[boeddeker]

The problem is that b_config may overwrite values in a_config (Basic assumption in sacreds config system). Hence, first b_config is executed and then a_config. So while b_config is executed, a is unknown.

Here, an example, that may give you an idea, how the sacred magic works:

ex = Experiment()

@ex.config
def a_config():
    b = 2
    a = 5 * b

@ex.named_config
def b_config(a):
    b = a / 2      # This cannot work, because a may depend on b
    b = 11      # This would work and a would then be 5*11

[mrzv]

I guess I don't understand the precedence. Named configs are executed before regular configs, but their values take precedence over regular configs? I.e., regular configs don't over-write the already set values? Is that right?

What is the right way to deal with this, where I want to access some global config value in a named config, which I promise won't modify it? Is there some kind of read-only access? Or that's impossible because the value is just not there? Do I have to copy the "default" values that everybody needs access too to every config? That seems like substantial code duplication.

[mrzv]

It seems having a global default_a variable works as intended. It does mean code duplication, but perhaps that's good enough for my purposes.

default_a = 5

@ex.config
def a_config():
    a = default_a

@ex.named_config
def b_config():
    a = default_a
    b = a / 2      # how to make this work?

[boeddeker]

I guess I don't understand the precedence. Named configs are executed before regular configs, but their values take precedence over regular configs? I.e., regular configs don't over-write the already set values? Is that right?

Yes, that is right. Sacred has priorities, which config values are used, if they are multiple times set. Additionally, there is some magic involved, when "config scopes" are used. This magic works only in one direction: higher priority -> lower priority

What is the right way to deal with this, where I want to access some global config value in a named config, which I promise won't modify it? Is there some kind of read-only access? Or that's impossible because the value is just not there? Do I have to copy the "default" values that everybody needs access too to every config? That seems like substantial code duplication.

I have never required something like that. I overwrite values in a named config (For me it is an "shortcut" for the CLI). Then the default config resolves dependencies (e.g. if statements). If you have precisely this use case, the global variable might be the best solution.

[mrzv]

The global variable seems to do the trick. Thanks for your help.

[mrzv]

Sorry, there is one more thing I don't understand. What's the logic why this doesn't work for sub-ingredients? If I explicitly mark model as dependent on data, as above, and then I want to access some variable in the data scope from a named_config for a model, why doesn't that work? Does the dependency not imply precedence?

[boeddeker]

I don't use ingredients, so I don't know.

[thequilo]

Named configs are always evaluated before "normal" configs, no matter where they are defined.

This is the place where it happens: https://github.com/IDSIA/sacred/blob/cd90ee1f55df04e2bb55d2e3b6badcd7c2a30bfe/sacred/initialize.py#L420-L442

While I agree that this is confusing and it would make total sense to fully evaluate the sub-ingredient's config (named + normal) before the parent ingredient, this would be a substantial change in the core code.

[thequilo]

And currently you can overwrite a sub-ingredient's config from its parent. Run with named:

from sacred import Experiment, Ingredient

i = Ingredient('i')


@i.config
def defaults():
    a = 42


ex = Experiment(ingredients=[i])


@ex.named_config
def named():
    i = {'a': 123}


@ex.automain
def main(_config):
    print(_config)

gives

{'i': {'a': 123}, 'seed': 515843388}

This is incompatible with the other processing order. But I'm unsure which one is more intuitive.