api
api copied to clipboard
materialsproject
[Bug]: mpr.materials.summary.search does not work for mp-api-0.45.8
Code snippet
What happened?
from mp_api.client.mprester import MPRester
mpr = MPRester('')
all_docs = mpr.materials.summary.search()
The code above can not work for mp_api-0.45.8. I tried to pip install and upgrade through python 3.12 and 3.13. Neither way works. The error messages are like follows:
Retrieving SummaryDoc documents:
210579/? [03:03<00:00, 1228.81it/s]
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[4], line 7
3 from copy import deepcopy
5 mpr = MPRester('HgxDEuZIOSRIDxNncSMKLVBi5NCk36W3')
----> 7 all_docs = mpr.materials.summary.search()
9 count = 0
11 for doc in all_docs:
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/mp_api/client/routes/materials/summary.py:367, in SummaryRester.search(self, band_gap, chemsys, crystal_system, density, deprecated, e_electronic, e_ionic, e_total, efermi, elastic_anisotropy, elements, energy_above_hull, equilibrium_reaction_energy, exclude_elements, formation_energy, formula, g_reuss, g_voigt, g_vrh, has_props, has_reconstructed, is_gap_direct, is_metal, is_stable, k_reuss, k_voigt, k_vrh, magnetic_ordering, material_ids, n, num_elements, num_sites, num_magnetic_sites, num_unique_magnetic_sites, piezoelectric_modulus, poisson_ratio, possible_species, shape_factor, spacegroup_number, spacegroup_symbol, surface_energy_anisotropy, theoretical, total_energy, total_magnetization, total_magnetization_normalized_formula_units, total_magnetization_normalized_vol, uncorrected_energy, volume, weighted_surface_energy, weighted_work_function, include_gnome, num_chunks, chunk_size, all_fields, fields, **kwargs)
359 query_params.update({"batch_id_not_eq": "gnome_r2scan_statics"})
361 query_params = {
362 entry: query_params[entry]
363 for entry in query_params
364 if query_params[entry] is not None
365 }
--> 367 return super()._search(
368 num_chunks=num_chunks,
369 chunk_size=chunk_size,
370 all_fields=all_fields,
371 fields=fields,
372 **query_params,
373 )
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/mp_api/client/core/client.py:1191, in BaseRester._search(self, num_chunks, chunk_size, all_fields, fields, **kwargs)
1169 """A generic search method to retrieve documents matching specific parameters.
1170
1171 Arguments:
(...) 1186 A list of documents.
1187 """
1188 # This method should be customized for each end point to give more user friendly,
1189 # documented kwargs.
-> 1191 return self._get_all_documents(
1192 kwargs,
1193 all_fields=all_fields,
1194 fields=fields,
1195 chunk_size=chunk_size,
1196 num_chunks=num_chunks,
1197 )
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/mp_api/client/core/client.py:1264, in BaseRester._get_all_documents(self, query_params, all_fields, fields, chunk_size, num_chunks)
1250 list_entries = sorted(
1251 (
1252 (key, len(entry.split(",")))
(...) 1259 reverse=True,
1260 )
1262 chosen_param = list_entries[0][0] if len(list_entries) > 0 else None
-> 1264 results = self._query_resource(
1265 query_params,
1266 fields=fields,
1267 parallel_param=chosen_param,
1268 chunk_size=chunk_size,
1269 num_chunks=num_chunks,
1270 )
1272 return results["data"]
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/mp_api/client/core/client.py:566, in BaseRester._query_resource(self, criteria, fields, suburl, use_document_model, parallel_param, num_chunks, chunk_size, timeout)
563 data = {"data": unzipped_data, "meta": {}}
565 if self.use_document_model:
--> 566 data["data"] = self._convert_to_model(data["data"])
568 data["meta"]["total_doc"] = len(data["data"])
569 else:
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/mp_api/client/core/client.py:1053, in BaseRester._convert_to_model(self, data)
1043 def _convert_to_model(self, data: list[dict]):
1044 """Converts dictionary documents to instantiated MPDataDoc objects.
1045
1046 Args:
(...) 1051
1052 """
-> 1053 raw_doc_list = [self.document_model.model_validate(d) for d in data] # type: ignore
1055 if len(raw_doc_list) > 0:
1056 data_model, set_fields, _ = self._generate_returned_model(raw_doc_list[0])
File ~/miniconda3/envs/py3.12/lib/python3.12/site-packages/pydantic/main.py:705, in BaseModel.model_validate(cls, obj, strict, from_attributes, context, by_alias, by_name)
699 if by_alias is False and by_name is not True:
700 raise PydanticUserError(
701 'At least one of `by_alias` or `by_name` must be set to True.',
702 code='validate-by-alias-and-name-false',
703 )
--> 705 return cls.__pydantic_validator__.validate_python(
706 obj, strict=strict, from_attributes=from_attributes, context=context, by_alias=by_alias, by_name=by_name
707 )
ValidationError: 8 validation errors for SummaryDoc
deprecation_reasons
Input should be a valid list [type=list_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/list_type
grain_boundaries
Input should be a valid list [type=list_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/list_type
es_source_calc_id.function-plain[validate()]
Value error, Invalid MPID Format [type=value_error, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/value_error
es_source_calc_id.int
Input should be a finite number [type=finite_number, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/finite_number
bandstructure
Input should be a valid dictionary or instance of BandstructureData [type=model_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/model_type
bulk_modulus
Input should be a valid dictionary [type=dict_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/dict_type
shear_modulus
Input should be a valid dictionary [type=dict_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/dict_type
has_reconstructed
Input should be a valid boolean [type=bool_type, input_value=nan, input_type=float]
For further information visit https://errors.pydantic.dev/2.11/v/bool_type
Version
mp-api-0.45.8
Which OS?
- [x] MacOS
- [ ] Windows
- [ ] Linux
Log output
Hi @Jeff-oakley, this should only be a problem with searching without filters, since this bulk downloads all data from AWS OpenData. Also make sure not to post your API key, you should regenerate a new on from your dashboard
@tsmathis looks like the OpenData entries got filled with NaN whenever a value was None.
Right now, a workaround is something like this:
from mp_api.client import MPRester
with MPRester(use_document_model=False) as mpr:
all_docs = mpr.materials.summary.search()
That will return a list of dictionaries. If you need the document model, then you can post-process them as follows:
from emmet.core.summary import SummaryDoc
import numpy as np
summary_docs = []
for i, doc in enumerate(docs):
try:
sd = SummaryDoc(**doc)
summary_docs += [sd]
except Exception as exc:
for k in list(doc):
if isinstance(doc[k],float) and np.isnan(doc[k]):
doc[k] = None
try:
sd = SummaryDoc(**doc)
summary_docs += [sd]
except Exception as exc_2:
summary_docs += [doc]
