figma-plugin icon indicating copy to clipboard operation
figma-plugin copied to clipboard

fix: rate limit GitLab requests when fetching tokens

Open cuserox opened this issue 5 months ago • 3 comments

Why does this PR exist?

Originally flagged by members of the community, where the plugin would intermittently fail to fetch all the tokens from their GitLab repository.

Specifically, the calls to the /projects/*/repository/files/*/raw endpoint in GitLab, which is used by the plugin when read()ing from the repository.

We use the @gitbeaker/rest library as a wrapper to interact with GitLab, in this case, the equivalent of calling the above endpoint is when this.gitLabClient.RepositoryFiles.showRaw(...) is invoked. This library offers a rateLimit configuration to be passed to the client.

Gathering data

Cloning the user's repository, there were around ~140 requests in .56s (equating to 250 rps) to the .../raw endpoint. GitLab's documentation states that "the rate limit on raw endpoints defaults to 300 requests per minute", the same as 5 rps.

What does this pull request do?

Adds a rateLimit configuration to the GitLab client when it's instantiated: '/projects/*/repository/files/*/raw': 4 (4 rps = 250 requests per minute)

The time that it took to wait for all raw files to be fetched below were calculated by using the native time method, 5 runs locally, like so:

console.time('this.gitlabClient.RepositoryFiles.showRaw');

const jsonFileContents = await Promise.all(jsonFiles.map((treeItem) => (
   this.gitlabClient.RepositoryFiles.showRaw(this.projectId!, treeItem.path, this.branch)
)));

console.timeEnd('this.gitlabClient.RepositoryFiles.showRaw');
  • Before rate limiting: average of 1.5s
  • After rate limiting: average of 1.9s

Testing this change

⚠️ Note: testing in my own cloned repository based on the user's tokens did not end up in timing out, even before rate limiting!

🤔 If someone can test before and after this change, for safety of not slowing down other users too much.

cuserox avatar Sep 05 '24 11:09 cuserox