Inferred parameters are missing from dataset even when explicitly added

[jenshnielsen]

# %%
from pathlib import Path

import numpy as np

import qcodes as qc
import qcodes.logger
from qcodes.dataset import (
    Measurement,
    initialise_or_create_database_at,
    load_or_create_experiment,
    plot_dataset,
)
from qcodes.instrument_drivers.mock_instruments import (
    DummyInstrument,
    DummyInstrumentWithMeasurement,
)
from qcodes.parameters import DelegateParameter


# %%
qcodes.logger.start_all_logging()

# %%
# configure the plots generated in this example to go to the example_output folder
qc.config.user.mainfolder = Path.cwd().parent / "example_output"

# %%
tutorial_db_path = Path.cwd().parent / "example_output" / "exp_output.db"
initialise_or_create_database_at(tutorial_db_path)

# %%
# preparatory mocking of physical setup
dac = DummyInstrument("dac", gates=["ch1", "ch2"])
dmm = DummyInstrumentWithMeasurement("dmm", setter_instr=dac)
del_param_1 = DelegateParameter("del_param_1", label="del param 1", source=dac.ch1)
del_param_2 = DelegateParameter("del_param_2", label="del param 2", source=dac.ch2)

# %%
tutorial_exp = load_or_create_experiment("doNd_VS_Measurement", sample_name="no sample")

# %%
# Setting up Measurement
meas = Measurement(name="1d_measurement of dmm from dac sweep", exp=tutorial_exp)
meas.register_parameter(dac.ch1)
meas.register_parameter(del_param_1, basis=(dac.ch1,))
meas.register_parameter(dmm.v1, setpoints=(del_param_1,))
meas.register_parameter(dmm.v2, setpoints=(del_param_1,))

# Running Measurement
with meas.run() as datasaver:
    for dac_sweep in np.linspace(0, 1, 10):  # sweep points
        del_param_1(dac_sweep)
        print(f"{dac_sweep} {dac.ch1()}")
        datasaver.add_result(
            (dac.ch1, dac.ch1()),
            (del_param_1, dac_sweep),
            (dmm.v1, dmm.v1()),
            (dmm.v2, dmm.v2())
        )

    dataset1 = datasaver.dataset

# %%
plot_dataset(dataset1)

# %%
dataset1.get_parameter_data("dac_ch1")

[samantha-ho]

I am fairly sure this bug stems from a logic issue in qcodes.dataset.data_set.py::DataSet._enqueue_results

        toplevel_params = set(interdeps.dependencies).intersection(set(result_dict))

        new_results: dict[str, dict[str, numpy.ndarray]] = {}

        for toplevel_param in toplevel_params:
            inff_params = set(interdeps.inferences.get(toplevel_param, ()))
            deps_params = set(interdeps.dependencies.get(toplevel_param, ()))
            all_params = inff_params.union(deps_params).union({toplevel_param})

In this fragment, inff_params is the set of parameters that the toplevel_param is inferred from. It will not include parameters that a member of the deps_params is inferred from.

However, since inferred_from can go in either direction from a deps_param (cf here) even iterating over them to get all of the inferences won't necessarily work for all cases covered in that design.

[samantha-ho]

There is also a further issue, that get_parameter_data will not display parameter data for inferred_from parameters by default. Since this function is used in to_xarray_dataset this also means that inferred_from data will not appear in the associated xarray dataset even if it is captured in the Qcodes dataset

This happens because get_parameter_data uses Interdependencies_.non_dependencies to identify top_level params, which will not include anything from the inferences dict.

[jenshnielsen]

@samantha-ho broke the last part out into #7062