Code resolving tag refs crashes in Linux

[dominique-pfister]

We're using isomorphic-git 1.7.8 with node 10.22.1. A CircleCI job running in a Linux docker image with 4 or 8 GB is killed because it runs out of memory. We have a piece of code that resolves git references in parallel. In a standalone script, this looks as follows:

const fs = require('fs');
const git = require('isomorphic-git');

/**
 * Returns the name of the current branch. If `HEAD` is at a tag, the name of the tag
 * will be returned instead.
 *
 * @param {string} dir working tree directory path of the git repo
 * @returns {Promise<string>} current branch or tag
 */
async function getBranch(dir) {
  // current branch name
  const currentBranch = await git.currentBranch({ fs, dir, fullname: false });
  // current commit sha
  const rev = await git.resolveRef({ fs, dir, ref: 'HEAD' });
  // reverse-lookup tag from commit sha
  const allTags = await git.listTags({ fs, dir });

  const tagCommitShas = await Promise.all(allTags.map(async (tag) => {
    const oid = await git.resolveRef({ fs, dir, ref: tag });
    const obj = await git.readObject({ fs, dir, oid });
    // annotated or lightweight tag?
    return obj.type === 'tag' ? {
      tag,
      sha: await git.resolveRef({ fs, dir, ref: obj.object.object }),
    } : { tag, sha: oid };
  }));
  const tag = tagCommitShas.find((entry) => entry.sha === rev);
  return typeof tag === 'object' ? tag.tag : currentBranch;
}

getBranch('.')
  .then((b) => { console.log(b); })
  .catch((e) => { console.log(e) });

The cloned git repo passed in dir has 414 tags, and is located here: https://github.com/adobe/helix-pages. I can reproduce the crash with above script in a docker image called circleci/node:10-stretch on my local Docker Desktop.

Note that the issue disappears when I change the await Promise.all(allTags.map(...)) construct into a for loop.

[stefan-guggisberg]

This issue boils down to the following:

Parallel calls to readObject() may lead to a race condition where the packfile index is read multiple times despite being cached, potentially causing an OOME: https://github.com/isomorphic-git/isomorphic-git/blob/main/src/storage/readPackIndex.js#L24-L33

While we've fixed the problem in our code by avoiding parallel readObject execution I think it would be worth considering guarding the cache lookup/population in above code with a Mutex.

[jcubic]

Hey, it seems that you know what you're doing if this is an issue for you maybe you can fix this?

Sorry for the late reply I've become a maintainer recently since the original author no longer has time to work on this project. But since I also don't have much time and don't know if I will be able to fix it myself.

[stefan-guggisberg]

Can't help I am afraid. I am pretty tied up with my current project, sorry.