pybliometrics icon indicating copy to clipboard operation
pybliometrics copied to clipboard

Suggestion for adding async via httpx or aiohttp

Open griff-rees opened this issue 4 years ago • 2 comments

This looks like a nice project. I think I'll stick with yours, but if my initial approach using async could be helpful I thought I'd flag that for discussion. Request quotas may mean it's not worth the complexity.

This gist is an example using httpx async features and aiofiles. Had aimed to use these async testing options. It's close but not quite the same as requests. I think aiohttp may be closer to requests.

griff-rees avatar Jun 26 '20 10:06 griff-rees

Hi @griff-rees ! I learned a lot through your post, having never heard of some of those packages.

Do I understand correctly that you're suggesting to use aiohttp or httpx instead of requests? What would be the benefits?

Michael-E-Rose avatar Jun 29 '20 15:06 Michael-E-Rose

It would be a significant change but I do think it would be compatible with python >= 3.6. I also think there were discussions to add async features to the next release of requests but that may be far off.

So: if scalability is a significant frustration for users of this library, even with the quotas, then it could be worth the effort. It may require detailed planning though, and like httpx it would be safest to retain a non-async option for compatibility.

griff-rees avatar Jun 29 '20 21:06 griff-rees