pelorus icon indicating copy to clipboard operation
pelorus copied to clipboard

Azure Boards performance improvement

Open weshayutin opened this issue 1 year ago • 1 comments

Fix

When we added 200 work items to our Azure DevOps test provider, we noticed that it was to slow to get all the data. Some rework was done, but there are still room for improvements.

Azure DevOps query language specs

Requirements

  • [ ] filter in query
    • [ ] filter projects in query as well This was not done because filtering by non-existing projects (wrong input data) breaks the query

    • run pytest exporters/tests/test_failure_exporter_azure_devops.py

      filter in query filter after query
      6.60s 8.46s
      6.59s 8.49s
      6.50s 8.20s
      6.25s 9.02s

      The results were faster 😄

  • [x] filter fields in query
    • run pytest exporters/tests/test_failure_exporter_azure_devops.py

      with filter without filter
      9.20s 9.06s
      8.70s 8.83s
      8.55s 8.38s
      8.79s 8.39s

      The results were not faster, but it does occupy less memory space.

weshayutin avatar Apr 11 '23 18:04 weshayutin

Moving out of iteration. The proper solution is to add in memory cache for all the failure exporters and not focus on just azure. The 2s save in favor of having invalid query is not worth the implementation if we can save almost 99% percent of the time with proper cache implementation. Note the cache is already present in the commit time exporter.

mpryc avatar Apr 14 '23 12:04 mpryc