lava-nc
Monitoring multiple Vars requires disabling checks for coherence between Process and ProcessModels
Dynamically adding new probes to a Monitor process (i.e. creating new RefPortand InPortsin the Monitor process) requires disabling checks for existence of corresponding LavaPyTypefor each newly created RefPorts/InPorts. More specifically the following lines in the compiler.py checks for this coherence between Process and ProcessModels:
b.set_variables(v)
self._get_port_dtype(pt, pm)
self._map_var_port_class(pt, proc_groups))
Could we make the Monitor process a hierarchical process, which instantiates additional sub processes (each with 1 RefPort + storage Var) for each Var we want to probe?
The Monitor process would store a list of Vars to probe + the parameters (dt, start, stop, buffer size, ...). The Monitor process model would then instantiate for each Var a SubProcess with a RefPort that gets connected to the Var and a storage Var, which stores the data each time step or whatever dt is.
This Subprocess does not change dynamically, only the number of Subprocesses we use changes, hence process and ProcessModel can be defined beforehand.
I agree this would not require this somewhat (currently) ugly disabling of checks we want to have in other cases. But wouldn't it be unnecessarily inefficient to spin up a system process for each individual Var we would want to monitor?
As we already discussed in another issue. I was hoping we could pull out those Proc/ProcModel consistency checks out from the Builder, which only gets executed at runtime, and do this upfront in the compiler where it might be easier to bypass in case of such special processes.
I thought the usage of RefPort/VarPort already motivated such a refactoring to do the LavaPyType validation earlier.
Well, the question is if we really want that. If people can add Vars and Ports dynamically it will lead to hardly understandable code which might be a nightmare to debug in the future.
Another idea would be to introduce a LavaType which represents a list and can be dynamically filled with a certain LavaType e.g. RefPorts. In this case it would be clear for the Process and ProcessModel that there is a list, which might be filled dynamically and the ProcessModels needs to handle that.
I also suggested a list LavaPyType at some point and I think it can be the cleanest solution. But I don't know if it would any undesired implications. I also agree with @PhilippPlank that dynamically creating Vars and Ports can make code hard to understand. Also agreed with @awintel that subprocess for each probe sounds inefficient.