Error with null references for get_papers

[alejosierra]

Bug description

Hi.

Thank you for a very useful library.

I am using the get_papers method to get the information about a batch of papers. I am also using the fields argument to, among others, return the references. However, the Paper class attempts to iterate a null list of references returned by the API for one of the papers resulting in a TypeError: 'NoneType' object is not iterable. I checked the direct API response and it returns "references": null for one of the papers in the json response.

Cheers,

Reproducible code example

from semanticscholar import SemanticScholar
sch = SemanticScholar()
ids_test = ['CorpusID:250295885', 'CorpusID:15467874']
fields_test = ["paperId", "externalIds","year", "references"]
papers_test = sch.get_papers(ids_query, fields=fields_test)

Error message

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[45], line 3
      1 ids_test = ['CorpusID:250295885', 'CorpusID:15467874']
      2 fields_test = ["paperId", "externalIds","year", "references"]
----> 3 papers_test = sch.get_papers(ids_query, fields=fields_test)

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/semanticscholar/SemanticScholar.py:170, in SemanticScholar.get_papers(self, paper_ids, fields, return_not_found)
    142 '''
    143 Get details for multiple papers at once
    144 
   (...)
    166 :raises: BadQueryParametersException: if no paper was found.
    167 '''
    169 loop = asyncio.get_event_loop()
--> 170 papers = loop.run_until_complete(
    171     self._AsyncSemanticScholar.get_papers(
    172         paper_ids=paper_ids,
    173         fields=fields,
    174         return_not_found=return_not_found
    175         )
    176 )
    178 return papers

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/nest_asyncio.py:98, in _patch_loop.<locals>.run_until_complete(self, future)
     95 if not f.done():
     96     raise RuntimeError(
     97         'Event loop stopped before Future completed.')
---> 98 return f.result()

File ~/anaconda3/envs/pybibenv/lib/python3.9/asyncio/futures.py:201, in Future.result(self)
    199 self.__log_traceback = False
    200 if self._exception is not None:
--> 201     raise self._exception
    202 return self._result

File ~/anaconda3/envs/pybibenv/lib/python3.9/asyncio/tasks.py:256, in Task.__step(***failed resolving arguments***)
    252 try:
    253     if exc is None:
    254         # We use the `send` method directly, because coroutines
    255         # don't have `__iter__` and `__next__` methods.
--> 256         result = coro.send(None)
    257     else:
    258         result = coro.throw(exc)

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/semanticscholar/AsyncSemanticScholar.py:214, in AsyncSemanticScholar.get_papers(self, paper_ids, fields, return_not_found)
    210 payload = { "ids": paper_ids }
    212 data = await self._requester.get_data_async(
    213     url, parameters, self.auth_header, payload)
--> 214 papers = [Paper(item) for item in data if item is not None]
    216 not_found_ids = self._get_not_found_ids(paper_ids, papers)
    218 if not_found_ids:

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/semanticscholar/AsyncSemanticScholar.py:214, in <listcomp>(.0)
    210 payload = { "ids": paper_ids }
    212 data = await self._requester.get_data_async(
    213     url, parameters, self.auth_header, payload)
--> 214 papers = [Paper(item) for item in data if item is not None]
    216 not_found_ids = self._get_not_found_ids(paper_ids, papers)
    218 if not_found_ids:

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/semanticscholar/Paper.py:144, in Paper.__init__(self, data)
    142 self._venue = None
    143 self._year = None
--> 144 self._init_attributes(data)

File ~/anaconda3/envs/pybibenv/lib/python3.9/site-packages/semanticscholar/Paper.py:373, in Paper._init_attributes(self, data)
    371 if 'references' in data:
    372     items = []
--> 373     for item in data['references']:
    374         items.append(Paper(item))
    375     self._references = items

TypeError: 'NoneType' object is not iterable

Package version

0.10.0

Python version

3.9.21

[danielnsilva]

Thanks @alejosierra. Code needs to ensure list integrity for some attributes.