Optimize the amount of data transferred to code-manager

[bobsilverberg]

There are a number of cases in Code Manager where we are receiving more information than we need in an API response. One example of this, which is already being addressed is documented in mozilla/addons#7369

Some general issues that have been identified are:

1. API responses contain data we never use. mozilla/addons#7369 is an example of that, but there may be others.
2. Duplicate data is being sent to a single "page" via multiple API requests.
3. Data that is cached in Code Manager is being received in subsequent API requests on different "pages".
4. Seemingly unnecessary API requests are being made.

This issue will be a tracker for specific instances of the above. We can add all of our ideas to this issue, and when we come up with something actionable we can open a separate issue for it.

Some specific ideas already discussed:

• The entire file tree is returned with every Browse and Compare request, but code-manager caches this for a given version, so we don't need to receive that data except for the first request made for a given version or given comparison. This is something that the client would need to determine, but we would also need API support to be able to request data for a new file in a version that would only return the data we need. This is an example of 3, above.
• We are requesting the Browse API from a Compare page. This is being done because the CodeOverview requires the contents of the current version of the file, but the compare API only returns the diff. This is an example of 4, above. Some possible solutions to this include returning the file content with the compare response, deriving the file content from the diff on the client, and/or changing the CodeOverview to use the diff instead of the file content.

We can add more ideas to this issue as they come up.

┆Issue is synchronized with this Jira Task

[bobsilverberg]

@diox Found it!

Here's a suggestion for a first step: I can look through code-manager and determine exactly what data is needed for different use cases, and then we can use that info to determine what changes we can make to APIs, which could include removing certain data from the responses, combining data from multiple APIs into a single one, etc.

WDYT?

[diox]

Sounds good. It's possible we end up finding that it's too difficult to untangle and that our time would better be spent elsewhere, but this will give us a good starting point.

[bobsilverberg]

I've finished looking through all of our code to see what data is used where. I am going to add some comments to this issue describing some of my findings, and we can then discuss what, if anything we want to do about them.

[bobsilverberg]

This was already discussed at https://github.com/mozilla/addons/issues/1822, but I think this could be a major gain:

We only need the file entries the first time we retrieve a version, after which we cache them, but every time a Browser or Compare API is called we get all of the entries back. After we have cached information for a version, we need much less information the next time we request a file for that version, so I am wondering if we should introduce an API that returns just information for a specific file. The code manager would request the current Browse or Compare API the first time a version is being loaded (and it could continue to include information for the current file, to avoid having to make an extra call), but for any subsequent files the code manager could call the new "file" API, which would only return the info needed for that one specific file.

@diox @EnTeQuAk What do you think?

[bobsilverberg]

An issue that discusses potential changes to the way the API deals with file entries has been opened at https://github.com/mozilla/addons/issues/7463.

[KevinMind]

Old Jira Ticket: https://mozilla-hub.atlassian.net/browse/ADDSRV-29