`PwBaseWorkChain`: `kpoints` override not passed in `get_builder_from_protocol()`

[mbercx]

Another input override which we fail to pass properly to from the overrides input argument in the PwBaseWorkChain.get_builder_from_protocol() method is the kpoints input.

Before closing this issue, we should probably do a proper check that all other inputs of the PwBaseWorkChain are properly passed when provided in the overrides. 😅

[mikeatm]

Has this been addressed yet? it would seem in this sample, the pseudo_family is not set, neither is the parameters according to the params_scf, and neither is the pseudo, whats the current state?

params_scf = {
        'CONTROL': {
            'calculation': 'scf',
            'verbosity': 'high',
            'wf_collect': True
        },
        'SYSTEM': {
            'ecutwfc': 60.,
            'nbnd': 150,
            'degauss': 0.22049585400,
            'starting_magnetization(1)': 0.10000000000,
        },
        'ELECTRONS': {
            'mixing_mode': 'plain',
        },
    }
params_dict = Dict(dict=params_scf)
kpoints = KpointsData()
kpoints.set_kpoints_mesh([5,5,5])
inputs =   {  
        'scf' : {'pw' : {'code': code, 'parameters': params_dict },  'pseudo_family': 'pseudo-dojo-oncv-fr', 'kpoints_distance':0.5 ,},
        'bands' : {'pw' : {'code': code, 'parameters': params_dict}, 'pseudo_family': 'pseudo-dojo-oncv-fr','kpoints_distance': 0.5 },
        'relax' : {'pw' : {'code': code, 'parameters': params_dict}, 'pseudo_family': 'pseudo-dojo-oncv-fr','kpoints_distance': 0.5 },
        'protocol': 'fast',
        'bands_kpoints_distance': 0.5
              }

builder= PwBandsWorkChain.get_builder_from_protocol(code,  structure_BiCaN, protocol="fast",  overrides=inputs, )
from aiida.engine import  run_get_node
res, node = run_get_node(builder)

a look at the result in the code at: this point in the base workflow shows the following: protocol=fast while, overrides={'pw': {'parameters': {'CELL': {'press_conv_thr': 0.5}}}} is the default for relax, while inputs has nothing to do with the above parameters: inputs={'clean_workdir': True, 'kpoints_distance': 0.5, 'kpoints_force_parity': False, 'meta_parameters': {'conv_thr_per_atom': 4e-10, 'etot_conv_thr_per_atom': 0.0001}, 'pseudo_family': 'SSSP/1.1/PBE/efficiency', 'pw': {'metadata': {'options': {'resources': {'num_machines': 1}, 'max_wallclock_seconds': 43200, 'withmpi': True}}, 'parameters': {'CONTROL': {'calculation': 'scf', 'forc_conv_thr': 0.001, 'tprnfor': True, 'tstress': True}, 'SYSTEM': {'nosym': False, 'occupations': 'smearing', 'smearing': 'cold', 'degauss': 0.01}, 'ELECTRONS': {'electron_maxstep': 80, 'mixing_beta': 0.4}, 'CELL': {'press_conv_thr': 0.5}}}} these appear nowhere in my input.

[mbercx]

Thanks for pinging this issue again, @mikeatm. I haven't come around to fixing the kpoints override yet, will try to prioritize this. A couple of comments on the code snippet you show:

1. The overrides input was designed to be a regular dict dictionary, without any AiiDA nodes. The conversion into nodes is done by the get_builder_from_protocol method. This is because the method relies on recursively merging the inputs with overrides, and using regular dictionaries also allows us to define both the protocol and overrides as YAML files. Point being: although I understand that your inclination might be to pass the parameters as a Dict node in the overrides, that will currently not work.
2. Take care with the structure of the overrides input, especially for more complex work chains like the PwBandsWorkChain. The 'relax' namespace of the PwBandsWorkChain exposes the inputs of the PwRelaxWorkChain, which in turn has both the base and base_final_scf top-level namespaces, which expose the inputs of the PwBaseWorkChain. So you're missing a 'base' level in the definition of your overrides (see updated example script below).
3. You're adding the code and protocol inputs in your overrides, but these are simply direct inputs to the get_builder_from_protocol method, and so don't need to be specified in the overrides.
4. Note that the pseudo_family needs to be one installed by aiida-pseudo. The package already has support for the pseudo-dojo pseudos (see example below).
5. You explicitly set 'starting_magnetization(1)': 0.10000000000,, but starting_magnetization can be better provided as a dictionary that maps the kind names to the starting magnetization values, e.g. 'starting_magnetization': {'Si': 0.2} for a silicon structure with only one kind. You also don't set nspin, which is set to 1 by default (non spin-polarised). The example below uses the spin_type input argument of the get_builder_from_protocol method to indicate that you want to run a collinear spin-polarised calculation.

I understand the protocol/overrides API isn't always easy to use, especially for more complex work chains. There is already an open issue (https://github.com/aiidateam/aiida-quantumespresso/issues/745) where with a short discussion on improving this. It might be easier to just obtain an initial builder from the method and then adapt it afterwards. However, I use the overrides a lot because this allows me to define the overrides I want as YAML files, see the example in the dropdown below.

Example of YAML file for `PwBandsWorkChain` overrides

clean_workdir: True
relax: null
scf:
  pseudo_family: &pseudo_family "SSSP/1.1/PBEsol/efficiency"
  pw:
    settings: &settings
      cmdline:
        - "-nk"
        - "16"
    metadata:
      options:
        account: &account 'mr0'
        resources: &resources
          num_machines: 1
          num_mpiprocs_per_machine: 128
          num_cores_per_machine: 128
        stash:
          source_list:
            - "out/aiida.save/charge-density.hdf5"
            - "out/aiida.save/charge-density.dat"
            - "out/aiida.save/data-file-schema.xml"
            - "out/aiida.save/paw.txt"
          target_base: "/scratch/project_465000028/charge_density"
          stash_mode: "copy"
bands:
  pseudo_family: *pseudo_family
  pw:
    settings: *settings
    metadata:
      options:
        account: *account
        resources: *resources

Below is a fully functional example based on the code snippet you provided:

from aiida import orm, load_profile
from aiida.engine import  submit
from aiida_quantumespresso.workflows.pw.bands import PwBandsWorkChain

load_profile('core')

code = orm.load_code('qe-7.0-pw@localhost')  # Be sure to update the label to your code!
structure = orm.load_node(2)  # This is trusty Si for my database, be sure to update the PK for yours!

params_dict = {
    'CONTROL': {
        'calculation': 'scf',
        'verbosity': 'high',
        'wf_collect': True
    },
    'SYSTEM': {
        'ecutwfc': 60.,
        'nbnd': 150,
        'degauss': 0.22049585400,
        'starting_magnetization': {'Si': 0.2}  # Setting to 0.2 to have a different value than the default
    },
    'ELECTRONS': {
        'mixing_mode': 'plain',
    },
}

# In case you want to override multiple `PwBaseWorkChain`s, I find it easier to define the base overrides once
base_overrides = {
    'pw': {
        'parameters': params_dict
    },
    'pseudo_family': 'PseudoDojo/0.4/PBE/FR/standard/upf',  # This is the label of the pseudo family installed by `aiida-pseudo install pseudo-dojo -r FR -f upf`
    'kpoints_distance': 0.4  # I set this to a different value than the 'fast' default
}

# And then use this dictionary for the various `PwBaseWorkChain` levels
overrides = {  
    'scf' : base_overrides,
    'bands' : base_overrides,
    'relax' : {
        'base': base_overrides
    },
    'bands_kpoints_distance': 0.5
}

from aiida_quantumespresso.common.types import SpinType

builder = PwBandsWorkChain.get_builder_from_protocol(
    code,  structure, protocol="fast", overrides=overrides, spin_type=SpinType.COLLINEAR,
)

builder.pop('nbands_factor')  # This is necessary because you explicitly set the bands in the overrides

I double checked, and the pseudo_family is overridden properly, as are the parameters and kpoints_distance. Give it a go with your structure and let me know if you have any questions. I'll get to fixing the kpoints override and converting some of this into documentation, which is admittedly also terribly outdated...

[mikeatm]

Thanks for pinging this issue again, @mikeatm. I haven't come around to fixing the kpoints override yet, will try to prioritize this. A couple of comments on the code snippet you show:

Thanks for the corrections, reading through the WF code its not immediately obvious what is going on.

'pseudo_family': 'PseudoDojo/0.4/PBE/FR/standard/upf',  # This is the label of the pseudo family installed by `aiida-pseudo install pseudo-dojo -r FR -f upf`

builder = PwBandsWorkChain.get_builder_from_protocol( ... builder.pop('nbands_factor') # This is necessary because you explicitly set the bands in the overrides

I double checked, and the `pseudo_family` is overridden properly, as are the `parameters` and `kpoints_distance`. Give it a go with your structure and let me know if you have any questions. ...

+1 i was able to get it to submit/run

converting some of this into documentation, which is admittedly also terribly outdated...

This will be very welcome. It took me a minute to understand there was something deeper going on after attempting to use this workflow.