jira icon indicating copy to clipboard operation
jira copied to clipboard

Using generator's to return results

Open blade2005 opened this issue 5 years ago • 2 comments

As a developer I'd like to be able to process results as they are received from the api calls for issue searches, rather than dealing with them after they've all been retrieved. This introduces a large memory cost when dealing with a large number of issues. Using a generator I can process each issue as it's returned allowing me to possibly thread handling each issue as it comes in.

I'd like it that when possible such as on JIRA.search_issues() that it return a generator using a yield statement.

blade2005 avatar Dec 16 '19 21:12 blade2005

I have the exact same concern. Using JIRA.search_issues() having all issues loaded into a list is a memory hog and limits the maximum number of issues you can process by the amount of memory available to the process. Having an option for returning a generator would make more sense.

zoltan-fedor avatar Jul 06 '20 16:07 zoltan-fedor

I want to look into implementing this, but I am currently unsure how to start. In #686 @ssbarnea said that the ResultList class should act as a Generator, but as far as I can see, the search_issues() method calls into _fetch_pages() which in turn first generates the whole Issue list before creating the ResultList.

I think, it would be easiest to modify _fetch_pages() to return a generator instead of the ResultList and then create the (extended with a __iter__() method) ResultList with that generator as argument. Would that be possible or am I thinking to complicated?

Independent of that, there is the untranslate feature in search_issues() that would also need to be adapted for the generator implementation, as it currently loops over all Issues in the ResultList.

dstengele avatar Jul 01 '21 15:07 dstengele