variable capture with multiple ingredients doesn't work

[ischlag]

Using 0.7b2. I have 2 ingredients each with individual named configs. Double annotations to capture both doesn't seem to work and leads to a non-deterministic error of unknown variables due to the failure of capturing. See the following example:

A.py

from sacred import Experiment
from model_loader import model_ingredient, get_model, data_ingredient

ex = Experiment("my_experiment", ingredients=[model_ingredient, data_ingredient])

@ex.automain
def run():
	model = get_model()
	print("model output: ", model.run())

model_loader.py

from sacred import Ingredient
import B
import C

## model specific configs
model_ingredient = Ingredient('model')

@model_ingredient.config
def cfg():
    model_name = "B"
    conf = "A"

model_ingredient.add_named_config("BA", {"model_name": "B", "conf": "A"})
model_ingredient.add_named_config("BB", {"model_name": "B", "conf": "B"})
model_ingredient.add_named_config("CA", {"model_name": "C", "conf": "A"})
model_ingredient.add_named_config("CB", {"model_name": "C", "conf": "B"})

## dataset specific configs
data_ingredient = Ingredient('data')
@data_ingredient.config
def cfg():
    dataset_name = "no data"

data_ingredient.add_named_config("X", {"dataset_name": "X"})
data_ingredient.add_named_config("W", {"dataset_name": "W"})

@model_ingredient.capture
@data_ingredient.capture
def get_model(model_name, conf, dataset_name): 
	print("load dataset: ", dataset_name)

	print("returns model " + model_name)

	if model_name == "B":
		return B.model(conf)
	elif model_name == "C":
		return C.model(conf)
	else:
		raise Exception("model doesn't exist?!")

And the two files B.py

class model:
	def __init__(self, conf):
		self.conf = conf

	def run(self):
		return "I'm model B with conf " + self.conf

... and C.py

class model:
	def __init__(self, conf):
		self.conf = conf

	def run(self):
		return "I'm model C with conf " + self.conf

Run like this:

python3 A.py with model.CA data.X

error:

WARNING - my_experiment - No observers have been added to this run INFO - my_experiment - Running command 'run' INFO - my_experiment - Started ERROR - my_experiment - Failed after 0:00:00! Exception originated from within Sacred. Traceback (most recent calls): File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/experiment.py", line 242, in run_commandline return self.run(cmd_name, config_updates, named_configs, {}, args) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/experiment.py", line 187, in run run() File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/run.py", line 190, in call self.result = self.main_function(*args) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/captured_function.py", line 47, in captured_function result = wrapped(*args, **kwargs) File "A.py", line 13, in run model = get_model() File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/captured_function.py", line 43, in captured_function bound) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/signature.py", line 104, in construct_arguments self._assert_no_missing_args(args, kwargs, bound) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/signature.py", line 161, in _assert_no_missing_args self.name, missing_args)) TypeError: get_model is missing value(s) for ['model_name', 'conf'] or

WARNING - my_experiment - No observers have been added to this run INFO - my_experiment - Running command 'run' INFO - my_experiment - Started ERROR - my_experiment - Failed after 0:00:00! Exception originated from within Sacred. Traceback (most recent calls): File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/experiment.py", line 242, in run_commandline return self.run(cmd_name, config_updates, named_configs, {}, args) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/experiment.py", line 187, in run run() File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/run.py", line 190, in call self.result = self.main_function(*args) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/captured_function.py", line 47, in captured_function result = wrapped(*args, **kwargs) File "A.py", line 13, in run model = get_model() File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/captured_function.py", line 43, in captured_function bound) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/signature.py", line 104, in construct_arguments self._assert_no_missing_args(args, kwargs, bound) File "/home/schlag/.virtualenvs/py3tf13/lib/python3.5/site-packages/sacred/config/signature.py", line 161, in _assert_no_missing_args self.name, missing_args)) TypeError: get_model is missing value(s) for ['dataset_name']

Also see this:

python3 A.py with model.CA data.X -p

Configuration (modified, added, typechanged, doc):
  seed = 110507708                   # the random seed for this experiment
  data:
    dataset_name = 'X'
  model:
    conf = 'A'
    model_name = 'C'
-------------------------------------------------------------------------------
WARNING - my_experiment - No observers have been added to this run
(...)

I solved this for now by moving the get_model function from model_loader.py to A.py and use ex.capture to capture the data and model dictionaries. But I wanted get_model to be in a different file than ex.

from sacred import Experiment
from model_loader import model_ingredient, data_ingredient

import B
import C

ex = Experiment("my_experiment", ingredients=[model_ingredient, data_ingredient])

@ex.config
def cfg():
	pass

@ex.capture
def get_model(model, data): # default value added to show running example
	print("load dataset: ", data['dataset_name'])

	print("returns model " + model['model_name'])

	if model['model_name'] == "B":
		return B.model(model['conf'])
	elif model['model_name'] == "C":
		return C.model(model['conf'])
	else:
		raise Exception("model doesn't exist?!")

@ex.automain
def run():
	model = get_model()
	print("model output: ", model.run())

[Qwlouse]

Yeah, capturing a function multiple times from different ingredients is not supported ATM. I hadn't thought of this usecase yet. I guess it could be implemented without too much trouble. But given that I'm rather busy these days, it is not likely to happen soon. Maybe you could make a load_dataset function that you capture from your data ingredient and call it from within your get_model function?