[fixed] Push list optimization

[vrigal]

The first patch was reverted as it was breaking up ingestion (certainly due to timeout because some pushes could be linked to thousands of commits).

I restored the limit of prefetched commits to 20, using a subquery. This was relatively easy to implement with django's ORM, but is not supported by MySQL:

This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery

The last alternative would be to perform the prefetch separately, then annotate the Push objects manually but this would be more of a hack.

Performances improvements still seems to be pretty good (if not even better):

• Small data (/api/project/autoland/push/?full=true&count=100&fromchange=<rev_id>)

  • master : 779 ms ± 63.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  • 7ad83d87: 688 ms ± 6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  • Approx 12% time reduction

• Large data (/api/project/autoland/push/?full=true&count=100)

  • master : 7.64 s ± 97.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  • 7ad83d87: 2.06 s ± 33 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  • Approx 73% time reduction

[codecov-commenter]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (be68fca) 76.86% compared to head (7ad83d8) 76.88%. Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7842      +/-   ##
==========================================
+ Coverage   76.86%   76.88%   +0.01%     
==========================================
  Files         538      538              
  Lines       26577    26597      +20     
  Branches     3359     3359              
==========================================
+ Hits        20429    20449      +20     
  Misses       5983     5983              
  Partials      165      165              

Files	Coverage Δ
tests/push_health/test_compare.py	100.00% <100.00%> (ø)
tests/webapp/api/test_push_api.py	100.00% <100.00%> (ø)
treeherder/push_health/usage.py	100.00% <100.00%> (ø)
treeherder/webapp/api/serializers.py	87.86% <100.00%> (-0.29%)	:arrow_down:
treeherder/webapp/api/push.py	58.29% <80.00%> (+0.18%)	:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.