gitlab4j-api icon indicating copy to clipboard operation
gitlab4j-api copied to clipboard

fetch projects in parallel

Open ramarro123 opened this issue 2 years ago • 0 comments

Hi,

i don't know if it's a feature that's possible to add, but with many projects if i do .getProjects() or getProjectsStream() or even lazy stream, what happen is that the projects are consumed from page 1 to 100 one request at time.

shouldn't be possible to create a method like getProjetcs(10,4) where 4 is the concurrency and fetch page 0,1,2,3 at the same time and then 4,5,6,7 and so on?

i know that the pagination limit is given via header with X-Total-Pages, so maybe the algo can do first fetch call 0, then based on total page open various thread to fetch them all in parallel?

it won't work with cursor pagination probably, but will speed up with this kind of paging? what do you think?

ramarro123 avatar Oct 20 '23 08:10 ramarro123