Searching with keywords

[dwaipayanroy]

trafficstars

Can we have a search mechanism for searching papers based on some keyword queries? There is already an API call for that: https://api.semanticscholar.org/graph/v1/paper/search?query=word+embedding+automatic+query+expansion&fields=title,abstract

[danielnsilva]

I've been working to deliver a new release based on the recent Semantic Scholar API upgrade, which includes searching for papers by keyword.

[wookayin]

Hi, I'm looking forward to this feature. Any chance you've gotten recently to wrap this feature up?

[danielnsilva]

Hi, @wookayin.

Unfortunately, I've been overloaded the last few months with other projects and haven't found enough time to work on this one.

But I have some code already done and will give my best to release a new version that enables the search feature soon. Maybe until next week.

[zouharvi]

Most of the work on this is probably figuring out the best API within this package. In the meanwhile, I've been using this little function to get the paper/search functionality:

from urllib.parse import quote
import urllib.request
import json

def paper_search(query, limit=10):
    query = quote(query)
    url_base = f"https://api.semanticscholar.org/graph/v1/paper/search?limit={limit}&fields=title,authors&query="
    url = url_base + query
    data = urllib.request.urlopen(url)
    data = data.read().decode("utf-8")
    return json.loads(data)["data"]

result = paper_search("Backtranslation Feedback Improves User Confidence")
for line in result:
    print(line)

Output:

{'paperId': 'f3c9084cd61dfcdc51c9d846a587863c17475540', 'title': 'Backtranslation Feedback Improves User Confidence in MT, Not Quality', 'authors': [{'authorId': '1429837660', 'name': 'Vilém Zouhar'}, {'authorId': '2072797005', 'name': "Michal Nov'ak"}, {'authorId': '1739254941', 'name': "Mat'uvs vZilinec"}, {'authorId': '151158933', 'name': 'Ondvrej Bojar'}, {'authorId': '2072784279', 'name': "Mateo Obreg'on"}, {'authorId': '3252072', 'name': 'Robin L. Hill'}, {'authorId': '1948646', 'name': 'F. Blain'}, {'authorId': '2006017', 'name': 'M. Fomicheva'}, {'authorId': '2065700826', 'name': 'Lucia Specia'}, {'authorId': '78447404', 'name': 'Lisa Yankovskaya'}]}
{'paperId': '4d9a4b4ac353a999c4fba3925739c9ea226b2af1', 'title': 'Optimally-calibrated non-invasive feedback improves amputees’ metabolic consumption, balance and walking confidence', 'authors': [{'authorId': '2146108553', 'name': 'Lauren Chee'}, {'authorId': '143696395', 'name': 'G. Valle'}, {'authorId': '2179503901', 'name': 'Michele Marazzi'}, {'authorId': '103705041', 'name': 'G. Preatoni'}, {'authorId': '151407570', 'name': 'F. Haufe'}, {'authorId': '144114747', 'name': 'M. Xiloyannis'}, {'authorId': '3134357', 'name': 'R. Riener'}, {'authorId': '2008652', 'name': 'S. Raspopovic'}]}
{'paperId': '4106abc70ced2ff666e94ea9b668128b0f982cce', 'title': 'High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling', 'authors': [{'authorId': '2111187011', 'name': 'Yu Zeng'}, {'authorId': '145527707', 'name': 'Zhe L. Lin'}, {'authorId': '1768964', 'name': 'Jimei Yang'}, {'authorId': '1519062024', 'name': 'Jianming Zhang'}, {'authorId': '2177801', 'name': 'E. Shechtman'}, {'authorId': '153176123', 'name': 'Huchuan Lu'}]}
...

[danielnsilva]

I think your viewpoint is correct on that @zouharvi . I am currently working on how to deal with paginated results. I intend to allow the user to iterate over results regardless number of pages.

[danielnsilva]

After a long delay, the new version was released and the search feature is finally here. I hope it could be helpful.