[Bug]: requesting all fields yields to fields with fields_not_requested, even when manually inputting them

[msiron-entalpic]

Code snippet

task_doc = mpr.materials.tasks.search(['mp-2022247'], fields=mpr.tasks.available_fields)[0]

What happened?

Running: task_doc.fields_not_requested

Returns:

['builder_meta',
 'nsites',
 'elements',
 'composition',
 'composition_reduced',
 'formula_anonymous',
 'volume',
 'density',
 'density_atomic',
 'symmetry',
 'tags',
 'dir_name',
 'state',
 'structure',
 'task_type',
 'included_objects',
 'vasp_objects',
 'task_label',
 'author',
 'icsd_id',
 'additional_json',
 'custodian',
 'analysis']

Even though all of these fields are available in mpr.tasks.available_fields however the REST API also do not have values for these fields at the /materials/tasks endpoint, but at the /material/core endpoint for material ID (mp-1187313) returns:

{"data":
{"calc_types":{"mp-1187313":"GGA Structure Optimization","mp-1403362":"GGA Static","mp-1758300":"GGA NSCF Uniform","mp-1789082":"GGA Static","mp-1626660":"GGA NSCF Uniform","mp-2020910":"R2SCAN Structure Optimization","**mp-2022247":"PBESol Structure Optimization",**"mp-2100559":"GGA NSCF Uniform","mp-2308485":"GGA NSCF Uniform"},
"task_types":{"mp-1187313":"Structure Optimization","mp-1403362":"Static","mp-1758300":"NSCF Uniform","mp-1789082":"Static","mp-1626660":"NSCF Line","mp-2020910":"Structure Optimization",**"mp-2022247":"Structure Optimization"**,"mp-2100559":"NSCF Line","mp-2308485":"NSCF Line"},
"run_types":{"mp-1187313":"GGA","mp-1403362":"GGA","mp-1758300":"GGA","mp-1789082":"GGA","mp-1626660":"GGA","mp-2020910":"R2SCAN",**"mp-2022247":"PBESol"**,"mp-2100559":"GGA","mp-2308485":"GGA"}}}}

I realize that the tasks endpoint does not have the data but the other endpoint does, should the data be harmonized with the MP API?

Version

mp_api: '0.42.2', pymatgen: '2024.9.17.1', emmet: '0.82.2'

Which OS?

• [X] MacOS
• [ ] Windows
• [ ] Linux

Log output

No error logs

[tschaume]

Thanks for reporting this, @msiron-entalpic! With the newest data release and mp-api client, the code snippet

fields = mpr.materials.tasks.available_fields
task_doc = mpr.materials.tasks.search(["mp-2022247"], fields=fields)[0]
task_doc.fields_not_requested

returns

['builder_meta', 'tags', 'dir_name', 'additional_json', 'custodian']

While these fields are part of available_fields and were requested, they are not present in the raw task doc returned from the database, so they get "lumped" in with fields_not_requested. Technically, they're simply "unset" but that distinction is not made when fields_not_requested is set by mp-api.

I've made a note in 87c072e1 and will try to come up with a fix when I get the chance. HTH.