scava
scava copied to clipboard
Delta creation getting out of hand on large projects
I recently pushed a commit (d3f855e) that details how much time it takes to compute the deltas for VCS / BTS / communication channels. Here's the result for https://github.com/elastic/elasticsearch (one of the project used in Bitergia's use case):
INFO [ProjectDelta (elasticsearch,20190101)] (14:47:12): Created Delta (vcs:4766ms, communications:1ms, bugs:1796820ms)
This is ~30min for the BTS. The delta creation seems to iterate on a lot of issues:
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/36265/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/36263/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/29963/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/16654/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/36251/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/29970/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/27312/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/36258/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/36256/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/30957/comments?per_page=100&page=1 )
AbstractInterceptor.intercept( https://api.github.com/repos/elastic/elasticsearch/issues/17997/comments?per_page=100&page=1 )
I understand that the poor performance is probably due to the large amount of data on this project + rate limitation on GitHub's APIs; but would there be any way to improve this?
Also, is there a way to not print these AbstractInterceptor.intercept()
debug messages to the console? Thanks!