EAMxx namelist/buildnml/atmchange remaining tasks and wishlist

[bartgol]

I am opening this issue to track stuff that still needs to be done for our handling of namelist in v1. Feel free to add items that you think are needed or simply desirable.

In the following, def.xml is the XML file with our defaults, while eam.xml is the intermediate xml file generated by buildnml.

• [x] Store the timestamp of the case XML files in eam.xml file. When buildnml runs, compare the current timestamp of those files with the one stored in eam.xml (if this file is already present). If some timestamp differ, nuke eam.xml and re-generate it. Bonus: store all the atmchange issued by the user in eam.xml, so that, if we have to re-generate it, we can re-issue them.
• [x] Currently, atm_procs_list for the whole atm is a "locked" entry, meaning it can be changed via atmchange. However, one may want to swap a parametrization for another, without having to create a new compset. Or one might want to remove a parametrization, without the need to add a new compset. It might be therefore desirable to allow to modify this entry. However, modifying it, one might add an atm proc that was not there, and whose params didn't make it to eam.xml. Therefore, if one modifies this entry, we should re-generate the section atmosphere_processes, parsing def.xml to grab all needed defaults.
• [ ] As EAMxx grows, it is possible that some parameters don't make it into eam.xml, but they are still used. This is probably a bug in buildnml, but in some corner cases, it might be needed. E.g., during development, one may want to try to make a new parameter configurable for a certain parametrization. Of course, the cleanest way to do this, would be to add the parameter to def.xml, but maybe for initial development, it's ok to just let the param in the input files without changing the defaults file. So we may consider adding a feature to atmchange to add entries, along the line of atmchange --add [ns1::ns2..::nsN::]param_name=XYZ, where [ns1::...::nsN::] is needed only if param_name does not uniquely identify the parameter in eam.xml (just like it is already the case in atmchange and atmquery). I'm not clear if atmchange should error out if the parameter already exists, or simply revert to a normal atmchange call in that case.
• [ ] Maybe add a --remove to atmchange, to remove some parameters. I'm not sure if this would make any difference, or, more correctly, whether it should. It would make a difference if the code already implements some default behavior if a parameter/parameterList is not found, in which case, removing the entry from eam.xml might allow to achieve that. But if that's the case, maybe we have a confusing code, where the def.xml is not prescribing the "default" behavior of the code.
• [ ] Allow to use atm parameters as selectors, to avoid injecting ATM vars in the CIME xml settings. That is, allow the following snippet in our xml defaults file

<param_1>one</param_1>
<param_2 param_1="one">foo</param_2>
<param_2 param_1="two">bar</param_2>

Edit: perhaps the 2nd point can be fixed (at least for the atm_proc_list part) by keeping the defaults of all atm procs in the intermediate file.