intake-esm

intake-esm copied to clipboard

Is your feature request related to a problem? Please describe. When using the catalog with a derived variable registry and calling to_dataset_dict, all variables that were needed for the derived variable computation are in the final datasets, even if they were not requested directly.

Describe the solution you'd like In the following imaginary example, say we have datasets for tasmin, tasmax and pr and a function for tas = func(tasmin, tasmax) in the DVR.

subcat = cat.search(variable=['tas', 'pr']) 
subcat.to_dataset_dict()

I would expect the ouput dataset to only include tas and pr, but with the current intake-esm tasmin and tasmax are also returned.

Describe alternatives you've considered

1. Wrapping it myself. Remembering what I wanted in a list and dropping the rest.
2. Instead of passing _requested_variables to the ESMDataSource objects, perform that subset in to_dataset_dict directly, after the derived variables have been created.
3. Add an aditionnal _needed_variables that would store a list of variables including dependencies, while _requested_variables would not list dependencies. And a subset would be performed by to_dataset_dict after the derived variable creation.

There is already a dependents variable created during the search() function, but it gets merged into _requested_variables and the information is lost afterwards.

Would it be a realistic solution to decouple _dependent_variables and _requested_variables, using both when necessary to open the datasets and extract data, and then only keep the _requested_variables for the final output?