SPHInX interface: passing changes to generic input down the line to SPHInX input tree

[freyso]

Summary

Once the SPHInX input tree (job.input.sphinx) is created, changes to input top-level settings have no effect. This could be fixed by registering call-backs when modifying input top-level settings.

Detailed Description

The current process of generating the SPHInX input tree (job.input.sphinx) at some not-so-clear point during job setup makes it difficult to ensure that all settings are correctly applied - in practice, the user may need to respect a particular order of settings to get the desired result.

Example:

job_bs = job_charge.restart_from_charge_density (...)
job.input["nSloppy"] = 6

The nSloppy setting has no effect, because restart_... comes with a complete tree.

Suggested solution:

1. Overload the __set_item__ function of job.input by something that supports key-based callbacks.
2. Register callbacks for generic settings that influence the SPHInX input tree while creating the latter
3. The callbacks could either auto-update or warn on mismatch.

Technical sketch

• input would get a dict _callbacks that maps keys to a list of callbacks. Before setting a new value for a key, _callbacks is checked for an entry and if available, the callbacks are called
• callback signature would be (input object, key, value)
• callbacks can use exceptions to modify setting behavior:
  • StopIteration if no further callbacks should be processed
  • LookupError (or KeyError?) if callback should be removed from callback list
  • any other Exception to prevent setting of the item
• have a standard update callback class (callable) with the following behavior
  • has a member tree-path to the affected setting within the input.sphinx hierarchy
  • tree-path in combination with the first argument of the callback allows to access the actual setting
  • checks if this setting (still) exists. If not (and not set to "auto-generate"), it will throw LookupError to remove the callback
  • compares the current value of the tree-path setting with the new value. If same, will return
  • updates the tree-path setting with the new value and returns
  • if the tree-path has not a unique end-point (e.g., if several groups with the same name exist), it will affect all endpoints
• installing the update could be wrapped to a nice-looking function, e.g.,

   self.input.link_setting ("nSloppy", "main/.../scfDiag/blockCCG/nSloppy")

• in principle, it would be nice to extend this also to the _generic_input settings, but then the original and linked setting live in different containers, so it may be a bit more painstaking to pass the correct container if things are copied around. In the worst case, one may need to make sphinx_job.input live inside _generic_input rather than next to it, and make input a property to maintain the external interface
• probably need some mechanism to make callbacks savable to hdf5. Probably as an additional (optional) property of the callback object that generates hdf-savable output, which could then be processed by a callback factory upon reading?

My concerns Are there any objections against such a mechanism? Is there any better (less complex) solution that would maintain the flexibility of access to all SPHInX features (input tree) while maintaining consistency with generic DFT settings? Is this compatible with pyiron philosophy? A possible alternative would be to introduce "link" objects into the SPHInX input tree that would be resolved at writing time. However, this would make conditionally generated parts of the input tree a lot more complicated.

[pmrv]

I find this elegant and wouldn't object if you built this, but it does seem a bit complex for an issue that we should have in every code. Why not maintain two input trees, one that is generated and modified by generic dft pyiron methods and attributes and one that is editable by the user. At write time the second tree is overlaid onto the first tree (effectively merging both trees, but giving precedence to the user tree). @samwaseda said once this wouldn't work but I didn't understand why.

Actually how come we do not have this kind of problem in vasp/lammps etc?